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Abstract 


NASA’s Air Traffic Management Technology Demonstration - 1 (ATD-1) will operationally 
demonstrate the feasibility of efficient arrival operations combining ground-based and airborne 
NASA technologies. The ATD-1 integrated system consists of the Traffic Management Advisor 
with Terminal Metering which generates precise time -based schedules to the runway and merge 
points; Controller Managed Spacing decision support tools which provide controllers with speed 
advisories and other information needed to meet the schedule; and Flight deck-based Interval 
Management avionics and procedures which allow flight crews to adjust their speed to achieve 
precise relative spacing. Initial studies identified air-ground challenges related to the integration 
of these three scheduling and spacing technologies, and NASA’s airborne spacing algorithm was 
modified to address some of these challenges. The Research and Procedural Testing of Routes 
human-in-the-loop experiment was then conducted to assess the performance of the new spacing 
algorithm. The results of this experiment indicate that the algorithm performed as designed, and 
the pilot participants found the airborne spacing concept, air-ground procedures, and crew 
interface to be acceptable. However, the researchers concluded that the data revealed issues with 
the frequency of speed changes and speed reversals. 
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1 Introduction 


Over the next twenty years, air traffic demand is predicted to increase by more than two percent 
per year [1], Arrivals into high-density airports, especially during peak traffic periods and 
inclement weather, often experience significant inefficiencies due to the use of miles-in-trail 
procedures and step-down descents. Use of these current procedures contributes to reduced airport 
throughput, increased controller workload, increased arrival delay, and increased aircraft fuel bum, 
emissions, and noise. Although advanced arrival procedures are available, they are underutilized 
due to the lack of supporting technology. 

The National Aeronautics and Space Administration’s (NASA) Air Traffic Management 
Technology Demonstration - 1 (ATD-1) combines advanced ground-based and airborne 
scheduling and spacing technologies in order to enable efficient arrival operations in high-density 
terminal airspace [2] [3]. The ATD-1 integrated system consists of three core components. 

• Traffic Management Advisor with Terminal Metering (TMA-TM) generates precise time- 
based schedules to the runway and merge points within the terminal area. 

• Controller Managed Spacing (CMS) decision support tools provide controllers with speed 
advisories and other information needed to meet the schedule. 

• Flight deck-based Interval Management (FIM) avionics and procedures allow flight crews 
to autonomously adjust their speed to achieve precise relative spacing. 

The Traffic Management Advisor (TMA) was originally developed at NASA Ames Research 
Center (ARC) and is currently used at Air Route Traffic Control Centers (ARTCC) nationwide to 
determine an appropriate arrival schedule [4], TMA-TM is an enhanced form of TMA that includes 
terminal area metering and enables the use of more efficient arrival procedures. CMS decision 
support tools, also developed at NASA ARC, provide controllers with the information necessary 
to achieve arrival schedule conformance using speed commands, thus reducing the use of tactical 
vectoring [5] [6]. The use of TMA-TM in conjunction with CMS tools has been assessed, and 
results indicate an increase in airport throughput [7] [8] [9] [10], 

Interval Management (IM) is an airborne spacing concept in which the flight crew is responsible 
for flying their aircraft at a speed that achieves their assigned time -based spacing interval behind 
a target aircraft, while Air Traffic Control (ATC) remains responsible for ensuring that all aircraft 
maintain safe separation. In normal operations, ATC adds a spacing buffer to the separation 
requirement to ensure that separation is always maintained. The goal of airborne spacing is to 
decrease this spacing buffer by decreasing the variability of the time error associated with an 
aircraft’s arrival at a specific point along its arrival route. The precise merging and spacing enabled 
by FIM avionics and flight crew procedures reduces excess spacing buffers and results in higher 
terminal throughput. Studies by MITRE [11] [12] [13], EUROCONTROL [14] [15] [16] [17], and 
NASA Langley Research Center (LaRC) [18] [19] [20] have demonstrated an increase in 
efficiency through the use of FIM operations. 

In addition to utilizing these advanced technologies in ATD-1, aircraft will fly new, more direct 
Area Navigation (RNAV) routes that extend from en route airspace to the runway. Consistent use 
of fuel efficient descents on RNAV arrivals rather than the step-down descents used today is a key 
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aspect for enabling broader use of Performance-Based Navigation (PBN). The Automatic 
Dependent Surveillance - Broadcast (ADS-B) infrastructure currently being implemented by the 
Federal Aviation Administration (FAA) will also be leveraged. The FIM tools will calculate speed 
commands using information provided by ADS-B, which is more accurate than traditional radar. 
The ability of flight crews to make more precise speed adjustments will enable a reduction in 
spacing buffers resulting in higher terminal throughput. 

The ATD-1 scheduling and spacing technologies have been evaluated independently, and each has 
demonstrated benefits. As an integrated system, these technologies will increase throughput, 
reduce delay, and minimize environmental impacts. Studies at NASA ARC and LaRC to 
demonstrate the ATD-1 concept and validate operational feasibility indicate that the concept is 
viable and operations are acceptable [21] [22] [23] [24] [25], A series of large-scale human-in-the- 
loop (HITL) simulations to evaluate system-level performance indicate that the ATD-1 ground- 
based technologies support PBN operations for high density traffic [26], 

These initial studies have also revealed a number of challenges associated with the integration of 
the ground-based and airborne technologies. During high demand operations, TMA-TM may 
generate a schedule and corresponding aircraft trajectories that include a substantial amount of 
delay in order to ensure aircraft separation and spacing. ATC will adjust aircraft speeds and/or 
vector aircraft off path to adjust arrival times to meet the schedule. The FIM algorithm calculates 
the estimated times of arrival (ETA) for the ownship and the target aircraft, and uses the difference 
to calculate a commanded speed. Since aircraft conducting FIM operations have no knowledge of 
the delay assigned to their target aircraft, they must use the published speeds to compute the target 
aircraft’s ETA. As a result, an aircraft performing FIM operations may follow a target aircraft with 
a TMA-TM generated trajectory that has substantial speed deviations from the speeds expected by 
the airborne spacing algorithm. 

NASA’s airborne spacing algorithm, Airborne Spacing for Terminal Area Routes (ASTAR), was 
modified to improve performance when following target aircraft with delayed trajectories. A batch 
simulation to investigate this new spacing algorithm with various delayed speed profiles and wind 
conditions indicated generally good performance. However, some types of target aircraft speed 
profiles were found to cause the spacing algorithm to command less than optimal speed control 
behavior [27]. 

To evaluate the performance of the new spacing algorithm with human pilots and controllers, the 
Research and Procedural Testing of Routes (RAPTOR) HITL simulation was conducted at NASA 
LaRC in 2014. The objective of this experiment was to assess if the new version of the ASTAR 
spacing algorithm (ASTAR-12) met the ATD-1 Project’s requirements for the following ATD-1 
Measures of Performance (MOP): 

• Percentage of controller-interrupted FIM operations (MOP 3.3.3), 

• Percentage of flight deck-interrupted FIM operations (MOP 3.4.3), 

• Flight crew acceptability of FIM operations (MOP 3.4.4), and 

• Flight crew workload during FIM operations (MOP 3.4.5). 

This paper describes the methodology and results of the RAPTOR HITL experiment. 
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2 Method 


2.1 Experiment and Scenario Design 

The focus of the RAPTOR experiment was to assess the performance of the new ASTAR-12 
airborne spacing algorithm. The airspace environment was modeled on the Albuquerque Center 
(ZAB) and Phoenix Sky Harbor (PHX) Terminal Radar Approach Control (TRACON) area. Each 
experiment scenario consisted of multiple air traffic flows involving 85 arrival aircraft flying the 
EAGUL FIVE and MAIER FIVE arrivals into PHX airport and landing on runway 26. Four of the 
arrival aircraft were flown by subject pilots operating as two-person crews. Two of these subject 
pilot crews flew full-scale, high fidelity fixed-base simulators, while the other two employed 
medium-fidelity desktop simulators. All four simulators were equipped with the ASTAR-12 
(version 12.02.34) airborne spacing algorithm and a flight deck control-display interface to enable 
FIM operations. Four additional arrival aircraft, which served as the target aircraft for FIM 
operations, were flown by four confederate pilots using medium-fidelity desktop simulators. The 
remaining 77 arrival traffic aircraft were flown by five pseudo-pilots, each of whom operated a 
single station that allowed him to control the basic functions of multiple aircraft. Five recently 
retired air traffic controllers served as Center (ZAB 43), Center (ZAB 39), Feeder, Final, and Ghost 
controllers issuing speed commands, vectors, and IM clearances. 

Subject aircraft performing FIM operations were expected to use the ASTAR-provided speed 
guidance whenever possible. This speed guidance is designed such that the spacing aircraft will 
achieve the assigned spacing goal behind the target aircraft at a predefined achieve-by point while 
remaining within 15% of the published RNAV arrival airspeed. In this experiment, the achieve-by 
point was the final approach fix (FAF). The FIM flight deck control-display interface shown in 
Figures 1 and 2 consisted of two side -mounted electronic flight bags (EFB) and configurable 
graphics display (CGD) devices mounted on the left and right outboard panels in the pilot’s 
forward field-of-view. 

During the RAPTOR experiment, the flight crew initially programmed ownship information into 
the EFBs. Confederate air traffic controllers issued the IM clearance via radio to the flight crew, 
who then entered the following information into the EFBs: 

• Assigned spacing goal (spacing interval required at the IM achieve-by point), 

• Target aircraft call sign, and 

• Target aircraft flight path (arrival and transition). 

After activating the FIM avionics, the flight crew flew the IM commanded speeds to achieve a 
precise spacing interval at the FAF. 
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Figure 2. Electronic flight bag 


Two flight scenarios were defined using the 1x2 experiment matrix shown in Figure 3 to allow an 
examination of two different wind fields based on actual winds recorded at PHX. The winds fields, 
a pairing of truth and forecast winds, were selected from a set of days that were used across the 
ATD-1 simulations and had been previously identified as stressing conditions. The first wind field 
was from 01/10/201 1, while the second was from 06/12/2011. Wind error has been shown to excite 
the groundspeed feedback term that was added to the ASTAR-12 spacing algorithm [28], and so 
these two winds were carefully chosen in order to examine the spacing performance of the FIM 
aircraft. The wind 01/10 scenario included very strong wind conditions at altitude and light to 
moderate wind at the airport. Scheduling delays for the target and FIM aircraft in this scenario 
ranged from two to nearly five minutes. The wind 06/12 scenario included moderate winds at 
altitude with schedule delays ranging from 30 seconds to three minutes. 

Initial conditions of the aircraft were based on actual traffic recorded at PHX on 11/14/2011 at 
18:00. This West flow traffic scenario was also used in the CA-5.1 HITL simulation conducted at 
NASA ARC. Modifications to this traffic scenario were made based on subject matter expertise, 
and included the removal of turboprops on the SWIRL arrival, overflights, and departures. Only 
the North side of the PHX airport with arrivals to runway 26 was simulated during the RAPTOR 
experiment. Additional details regarding the design of the experiment scenarios and graphical 
representations of the standard terminal arrival routes (STARs) used during the experiment are 
included in Appendix A. 
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Forecast: 2011011 0/09Z Forecast: 2011061 2/05Z 

Truth: 2011011 0/06Z Truth: 20110612/02Z 


Wind 01/10 Wind 06/12 


Figure 3. Experiment design matrix 


2.2 Participants 

2.2.1 Subject Pilots 

Subject pilots consisted of four two-person crews of current, qualified 757/767, 777, and 787 pilots 
employed by major U.S. air carriers (i.e., a total of eight commercial airline pilots). All subject 
pilots were male and ranged from 38 to 64 years in age. On average, the pilots had 22 years of 
airline experience and over 12,750 hours of commercial airline flight time. To minimize potential 
effects associated with different airline operating procedures, all two-person crews were paired 
from the same airline, and the pilots flew in their current operational position (captain or first 
officer) using their company’s standard operating procedures modified to include FIM operations. 
These subject pilots participated in a two-day experiment session at NASA LaRC on February 5- 
6,2014. 

Data were also collected from six crews of current, qualified 757/767 and 777 pilots during two 
single day experiment sessions conducted in January 2014. However, data collected in January 
were not included in formal analyses since strict adherence to the RAPTOR experiment protocol 
was not possible due to unexpected weather-related closings of LaRC on January 29 and 30, a 
required software change implemented on February 3, and changes in controller behavior observed 
during the experiment sessions conducted in January as compared with those conducted in 
February. 

2.2.2 Confederate Pilots and Pseudo-Pilots 

Four confederate pilots utilized medium-fidelity desktop simulators to operate four human-piloted 
target aircraft for FIM operations. These pilots were male and ranged from 57 to 66 years in age. 
They had an average of 29 years and 16,000 hours of commercial airline experience. 

Five additional pseudo-pilots used desktop pseudo-pilot stations to operate 77 traffic arrival 
aircraft by responding to controller commands issued via radio communications. These pseudo- 
pilots were male and ranged in age from 61 to 70 years. On average, these pilots had 17 years of 
military flight experience, 19 years of commercial airline experience, and over 12,800 hours of 
commercial airline flight time. 
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2.2.3 Confederate Controllers 


Five recently retired air traffic controllers served as confederate controllers. The confederate 
controllers included two ZAB Center controllers (sectors 43 and 39), one PHX Feeder controller, 
one PHX Final controller, and one Ghost controller. The three en route controllers had 25-26 years 
of en route ATC experience at Oakland ARTCC. The Feeder and Final controllers had 34.5 and 
15 years of TRACON experience, respectively. All five controllers were trained at NASA ARC in 
the use of the ground-based tools and had participated in previous ATD-1 experiments. 

2.3 Experiment Protocol 

The subject pilots received training materials and had access to computer-based training prior to 
arriving at NASA LaRC. After arriving at LaRC, the pilots received six hours of classroom and 
hands-on training, including flying five training scenarios. The four two-person crews participated 
in a two-day experiment session and completed a total of four data collection flights. The first day 
of the experiment session began with classroom and hands-on training, and then one data collection 
flight was conducted. The second day consisted of the remaining data collection flights, followed 
by a post-experiment questionnaire and group debrief session. Each data collection flight lasted 
approximately 45 minutes, and pilots completed a post-run questionnaire following each flight. 
Post-run and post-experiment questionnaires were also administered to and feedback was collected 
from the confederate controllers. 

Each flight crew flew both experiment scenarios twice - once with the captain as the pilot flying 
(PF) and the first officer as the pilot monitoring (PM), and once with the first officer as the PF and 
the captain as the PM - for a total of four experiment runs. The run order was partially 
counterbalanced, and within each crew the pilots switched PF and PM responsibilities between 
runs. The Center and Ghost controllers also rotated positions between runs. Additional details 
regarding the daily schedule and experiment run order are included in Appendix B. 

2.4 Scheduling and Spacing Technologies 

This experiment utilized an integrated set of ground-based and airborne technologies consisting of 
TMA-TM, CMS decision support tools, and FIM avionics and procedures. These scheduling and 
spacing technologies are described below. 

2.4.1 Traffic Management Advisor with Terminal Metering 

TMA-TM is an extension of the operational TMA that determines an arrival schedule based on 
airport conditions, airport capacity, required spacing, and weather conditions. This scheduling tool 
uses the predicted trajectory of each aircraft along its projected arrival route to calculate the ETA 
and corresponding Scheduled Time of Arrival (STA) at various meter and merge points along the 
aircraft flight path. The schedule is then deconflicted at each of the scheduling waypoints by 
delaying aircraft until they are properly spaced. The TMA-TM schedule is broadcast to the en 
route and TRACON controller positions for use by the CMS tools to assist the controllers in 
maintaining optimum flow rates to the runways. 
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Figure 4. Traffic Management Advisor with Terminal Metering display 


The research version of TMA-TM (rTMA) utilized during the RAPTOR experiment had the 
display shown in Figure 4 with six timelines of which three were active. The second timeline 
showed the arrival schedule for PHX runway 26, and the fifth and sixth timelines showed the 
arrival schedule for two metering fixes around the PHX TRACON. The current clock time 
(1500:58) appears in the upper left comer. The white numbers located in the middle of each 
timeline indicate minutes after the hour. As time progresses, the timelines scroll toward the bottom 
of the screen with the current time at the bottom and one hour in the future at the top. On the left 
of each timeline, each aircraft’s ETA at the metering fix or mnway threshold is shown in green. 
On the right side of each timeline, the aircraft’s STA is shown in amber or cyan. As aircraft 
approach the airport, TMA continually recalculates the schedule based on the aircraft’s current 
ETA until the aircraft reaches a freeze horizon set at 160 NM from each metering fix. After 
crossing the freeze horizon, the schedule for each aircraft remains fixed. An amber call sign 
indicates that an aircraft is outside of the freeze horizon, and a cyan call sign indicates that it is 
past the freeze horizon. If the green indicator on the left side of the timeline is below the 
corresponding indicator on the right, the aircraft is early; otherwise, the aircraft is late. 
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2.4.2 Controller Managed Spacing 


CMS decision support tools provide TRACON controllers with the information needed to meet 
the TMA-TM generated schedule. The CMS tools used in the RAPTOR experiment were available 
to both the Feeder and Final controllers, and consisted of timelines, slot markers, early/late 
indicators, speed advisories, and sequence numbers (see Figure 5). 

The schedule timelines provide graphical depictions of an aircraft’s schedule conformance by 
displaying the relationship between each aircraft’s ETA and STA to a merge point or the runway 
threshold. If the ETA is earlier than the STA, the aircraft needs to absorb delay; if the ETA is later 
than the STA, the aircraft needs to be advanced. 

Slot marker circles indicate where an aircraft should be located at a given time if it were to fly the 
RNAV arrival, through the forecasted wind field, meeting all published speed and altitude 
restrictions, and arrive at its STA to the merge point or runway threshold. The relative position of 
the aircraft symbol and the slot marker provides a quick visual indication of how the aircraft is 
positioned relative to its STA. If the aircraft is on schedule, the aircraft symbol is centered within 
the slot marker. In this experiment, the diameter of the slot marker circle represented 15 seconds 
of flying time at the aircraft’s current ground speed. 

Early/late indicators in the aircraft full data block (FDB) enable controllers to quickly assess the 
schedule conformance information for that aircraft. These indicators represent the difference in 
minutes and seconds between the ETA and STA to the next merge point or the runway threshold. 
They are generated when a single speed advisory cannot be calculated to resolve schedule 
conformance with a 10 knot discrimination and the difference between the ETA and STA is greater 
than or equal to five seconds. 

Speed advisories in the aircraft FDB enable controllers to formulate speed clearances for aircraft 
not performing FIM operations. The speed advisory is a recommended calibrated airspeed (CAS) 
which is predicted to place the aircraft back on schedule before reaching the scheduling fix. Speed 
advisories are displayed when an aircraft’s ETA is more than five seconds earlier or later than its 
STA, and only if the predicted speed will resolve the difference between the ETA and STA by the 
next metering point; otherwise the early/late indicator is shown. 

Sequence numbers in the aircraft FDB display the number of the aircraft in sequence to the runway. 
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Figure 5. Controller Managed Spacing tools 


2.4.3 Flight deck-based Interval Management 

The FIM tools provide onboard speed guidance to the flight crew to achieve a precise spacing 
interval behind a target aircraft and meet the schedule set by TMA-TM. The FIM avionics include 
a spacing algorithm onboard the aircraft that uses information provided by ATC in the form of an 
IM clearance along with ADS-B data from the target aircraft to compute IM commanded speeds. 
The flight crew follows these speeds to achieve precise in-trail spacing at a designated achieve-by 
point. 

In order to perform FIM operations in this experiment, all FIM aircraft were equipped with NASA 
LaRC’s airborne spacing algorithm, ASTAR-12, and a flight deck control-display interface (see 
Figures 1 and 2). The ASTAR algorithm produces speed guidance by determining time-to-go until 
an aircraft and its target reach an achieve-by point along a 4-D trajectory. The procedure for 
conducting a FIM operation began when ATC provided a FIM-equipped aircraft with an IM 
clearance, which included the target aircraft’s call sign and route, and a spacing goal computed by 
rTMA. The flight crew then entered this information into the interface and followed the speeds 
provided by the algorithm to achieve the spacing interval at the FAF. 

During FIM operations, the EFB displayed the target aircraft call sign; the commanded speed 
required to achieve precise interval spacing behind the target aircraft; and a Fast/Slow indicator to 
provide trend information and guidance regarding required FIM aircraft decelerations and 
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accelerations to conform to the ASTAR algorithm. In order to provide the pilots with required 
information in the forward field-of-view, the CGD supplied the following five elements during a 
FIM operation: 

• FIM status indication of when the FIM aircraft was actively performing FIM operations, 

• Target aircraft call sign, 

• Commanded speed, 

• Fast/Slow indicator, and 

• System caution and information messages. 

Upon reaching the FAF, the information on the EFBs and CGDs was turned off, and the flight 
crew began decelerating to their landing speed. Appendix C illustrates the use of the EFB for data 
entry, activation, suspension, resumption, cancellation, and termination of FIM operations. 
Illustrations of the information elements presented on the CGD during FIM operations are also 
provided. 

In addition to the avionics onboard the FIM aircraft, several FIM indicators on the controller 
displays were also utilized during this experiment. FIM information displayed in the Meter List 
provided en route controllers with the information necessary to issue the IM clearance. This 
included the aircraft call sign, meter fix STA, meter fix delay, spacing interval in seconds, target 
aircraft call sign, and target aircraft route as shown in Figure 6. 



Aircraft 


Meter Fix 

Call sign 


STA 


Meter Fix 


Spacing 

Delay 


Interval 


Target Aircraft 


Target 

Call sign 


Aircraft 


Route 


Figure 6. FIM information displayed on Meter List 


FIM advisory displays to the en route controllers indicated the eligibility and status of the IM 
clearance. Aircraft that were FIM capable displayed a yellow symbol on top of the FDB. This 
was changed to a magenta “@” symbol after the controller issued the IM clearance, and to a 
magenta “S” once the aircraft reported paired spacing (see Figure 7). On the feeder and final 
controller displays, aircraft that had been issued an IM clearance displayed “FIM” in the FDB, and 
aircraft that were currently performing FIM operations displayed “SPC” in the FDB. These 
symbols are shown in Figure 8. 
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Figure 8. IM clearance status indicators for TRACON controllers 


2.5 Facilities and Equipment 

The RAPTOR experiment utilized two facilities at NASA LaRC: the Flight Simulation Facility 
(FSF) and the Air Traffic Operations Laboratory (ATOL). Descriptions of each facility and its 
equipment are provided in this section. 

2.5.1 Flight Simulation Facility 

During the RAPTOR scenarios, two of the four FIM aircraft were flown by subject pilots utilizing 
two of the full-scale simulators within the FSF - the Integration Flight Deck (IFD) and the 
Development and Test Simulator (DTS) [29], 
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2.5.1. 1 Integration Flight Deck 


The IFD is a full-scale simulator representative of a large commercial transport category aircraft 
and is driven by an appropriate aircraft dynamics mathematical model. The cockpit includes 
standard ship’s instruments representative of a line operations aircraft, and the cockpit’s visual 
system is a panorama system that provides 200° horizontal by 40° vertical field-of-view. During 
the RAPTOR experiment, one of the four subject flight crews flew the IFD, and the visual scene 
used was the PHX terminal environment in a daytime setting. As noted previously, the IFD 
simulator was equipped with the ASTAR algorithm and the prototype FIM crew interface to enable 
the flight crew to perform FIM operations. Figure 9 shows the positions of the EFBs and CGDs 
within the IFD. 


Captain’s electronic flight bag (EFB) and configurable graphics display (CGD) 



Figure 9. Integration Flight Deck equipped with the flight-deck control display interface 


2.5.1.2 Development and Test Simulator 

The DTS is a full-scale fixed-base simulator representative of a large generic commercial transport 
category aircraft, and is driven by a high-fidelity aerodynamic mathematical model. The cockpit 
displays are incorporated in four 17-inch displays and the visual system includes a 210 ° horizontal 
by 45 “vertical out-the-window field-of-view. In order to enable the flight crew to perform FIM 
operations, the DTS was equipped with the ASTAR algorithm, two side-mounted EFBs, and two 
CGDs rendered using portions of the simulator’s existing outboard heads-down displays (see 
Figure 10). 
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Captain’s electronic flight bag (EFB) and configurable graphics display (CGD) 



First officer’s configurable graphics display (CGD) and electronic flight bag (EFB) 


Figure 10. Development and Test Simulator equipped with the flight-deck control display interface 


2.5.2 Air Traffic Operations Laboratory 

The ATOL contains a network of hundreds of real-time, medium-fidelity aircraft simulators and 
utilizes a simulation platform, known as the Airspace and Traffic Operations Simulation (ATOS), 
which can be used for both batch and real-time HITL experiments. During the RAPTOR HITL 
experiment, two FIM aircraft, four target aircraft, and 77 traffic aircraft (total of 83 of the 85 
aircraft in each scenario) were simulated in the ATOL. ATC stations within the ATOL were also 
utilized to enable confederate air traffic controllers to provide a realistic ATC environment. 

2.5.2.1 ASTOR Stations 

Each desktop aircraft simulator within the ATOL is referred to as an Aircraft Simulation for Traffic 
Operations Research (ASTOR) [30], An ASTOR provides a medium-fidelity representation of a 
commercial transport aircraft, its flight deck systems, and the airborne components of a realistic 
future communications, navigation, and surveillance infrastructure. ASTORs can be configured 
for single-crew or dual-crew operations. 

Two of the four FIM aircraft flown by subject pilots during the RAPTOR experiment were 
simulated using two dual-crew ASTORs. The dual-crew ASTORs were utilized in order to provide 
the crew interaction that is critical in today’s airlines. Each of these desktop simulators contained 
a high-fidelity six degree of freedom dynamics model and aircraft displays shown on three 27-inch 
touchscreen monitors (Figure 1 1). Pilots interacted with the simulators through either a mouse or 
touchscreen interface. In addition to normal aircraft systems, the prototype FIM crew interface was 
emulated on the dual-crew ASTORs to enable flight crews to perform FIM operations. The four 
target aircraft flown by confederate pilots were simulated using four single-crew ASTORs. 
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Figure 11. Dual-crew ASTOR desktop simulator displays 


2 . 5 . 2. 2 MACS Pseudo-Pilot Stations 

The majority of the aircraft in the RAPTOR experiment were simulated using the NASA ARC 
Multi- Aircraft Control System (MACS) [31]. The use of MACS desktop pseudo-pilot stations 
allowed a single operator to control the basic functions of multiple aircraft. The 77 traffic aircraft 
were controlled by a group of five MACS pseudo-pilots, with three pilots working the ZAB high 
altitude sectors and two pilots working the PHX TRACON feeder and final sectors. Figure 12 
illustrates some of the items that can be shown on a MACS pseudo-pilot display. The pseudo-pilot 
can take control of any active MACS aircraft by selecting it from the aircraft list with a mouse 
click. The aircraft’s speed, heading, and altitude can be altered by entering changes in the MACS 
Mode Control Panel (MCP) or Multi-Purpose Control Display Unit (MCDU). 



Figure 12. MACS pseudo-pilot station display 
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2.5.2.3 ATC Stations 


ATC controller stations equipped with MACS were utilized within the ATOL to enable 
confederate air traffic controllers to provide a realistic ATC environment. All controller positions 
used standard Display System Replacement (DSR) or Standard Terminal Automation 
Replacement System (STARS) displays augmented with CMS tools. Five recently retired air 
traffic controllers served as confederate Center (ZAB 43), Center (ZAB 39), Feeder, Final, and 
Ghost controllers. 


3 Results and Discussion 

Quantitative data, audio and video recordings, and researcher observations were collected to assess 
the ATD-1 MOPs regarding the number of violations of separation, percentage of controller- 
interrupted FIM operations, FIM spacing goal conformance, and percentage of flight deck- 
interrupted FIM operations. To assess flight crew acceptability of FIM operations and flight crew 
workload of FIM operations, subjective response data were collected via electronic post-run 
questionnaires. The pilot and controller post-run and post-experiment questionnaires are included 
in Appendices D, E, F, and G. 

Due to the small sample size, data from the PF and PM were combined. Data were also combined 
across the three different simulator types (IFD, DTS, and dual-crew ASTORs). It should be noted 
that a simulation artifact caused inaccurate temperature modeling which may have affected the 
spacing, possibly resulting in inaccurate spacing intervals and violation of separation criteria, 
particularly for the wind 06/12 scenario. 

3.1 Number of Violations of Required Separation (MOP 3.2.1) 

The separation violations recorded during the experiment are shown in Table 1. The first two 
columns in the Table indicate the run number and wind field of the scenario. The next four columns 
show the call sign and arrival route of the aircraft involved in the separation violation. The last 
four columns show the required horizontal separation, actual minimum horizontal and vertical 
separation, and location where the violation occurred. There were nine violations in total, eight of 
which occurred during the wind 06/12 scenario. Eight of the nine separation violations involved 
aircraft flown by the subject pilots. 

During Run 1, the flight crew in NASA02 did not comment on the separation violation. The 
violation involving NASA05 corresponds to the flight deck-interrupted FIM operation discussed 
in Section 3.4. Additional details regarding pilot perceptions and events that occurred during the 
violation involving NASA06 in Run 1 are discussed in Section 3.5. The subject pilots flying 
NAS AO 1 during Run 3 did not mention the separation violation, but one did co mm ent on receiving 
many speed changes at low altitude, including a speed reversal. During Run 4, none of the flight 
crews commented on the separation violations. One of the pilots in NASA05 did mention that he 
had to configure the aircraft much earlier than he would have during an operational situation. 
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Table 1. Violations of required separation 








Horizontal 

Min 

Min 








Sep Req 

Horizontal 

Vertical 

General 

Run 

Wind 

Aircraft 1 

Route 1 

Aircraft2 

Route2 

(NM) 

Sep (NM) 

Sep (ft) 

Location 



NASA79 

MAIER 

NASA02 

MAIER 

4.0 

3.5 

1300 

On Final 



NASA71 

MAIER 

NASA06 

EAGUL 

2.5 

2.3 

800 

On Final 

1 

06/12 

NASA77 

EAGUL 

NASA05 

MAIER 

2.5 

2.4 

820 

On Final 



DCM1208 

EAGUL 

SKW4501 

MAIER 

3.0 

2.9 

863 

Turn onto 
Final 

3 

01/10 

NASA79 

MAIER 

NAS A0 1 

MAIER 

2.5 

2.4 

820 

On Final 



NASA79 

MAIER 

NASA02 

MAIER 

4.0 

3.5 

1300 

On Final 

A 

06/12 

NASA78 

EAGUL 

NAS A0 1 

EAGUL 

4.0 

3.6 

1300 

On Final 

4 

NASA71 

MAIER 

NASA06 

EAGUL 

2.5 

2.3 

800 

On Final 



NASA77 

EAGUL 

NASA05 

MAIER 

2.5 

2.3 

800 

On Final 


3.2 Percentage of Controller-Interrupted FIM Operations (MOP 3.3.3) 

The number of controller-interrupted FIM operations was determined from the video and audio 
recordings, in addition to researcher observations. The percentage of controller-interrupted FIM 
operations was then computed for each scenario by 

Percentage of controller-interrupted FIM operations 

= 100% * number of controller-interrupted FIM ops / total number of FIM ops. 

There was only one controller-interrupted FIM operation (see Table 2), which occurred during the 
first run of the wind 06/12 scenario. NASA01 was performing FIM operations during hand-off to 
final approach. The flight crew checked-in, informed the final controller that they were paired 
behind their target aircraft, and reported their current speed. The final controller interpreted this 
reported speed as having been previously commanded by the feeder controller, and responded 
“NASA01, Phoenix Approach, you are cleared runway 26 ILS approach resume, correction, fly 
normal speeds.” The final controller intended the aircraft to resume FIM operations and fly the IM 
commanded speeds. However, the flight crew interpreted “fly normal speeds” as terminating FIM 
operations, so they flew the published approach for the remainder of the flight. This 
miscommunication resulted in an unintentional controller-interrupted FIM operation. 

In order to assess the following a priori hypothesis, 

Hypothesis 1: The rate of early FIM termination by ATC is less than 30%, 

statistical analysis was performed using the binomial test of one proportion [32] to test whether 
the percentage of controller-interrupted FIM operations was less than or equal to 30% vs. the 
percentage was more than 30%. For both scenarios, the percentage of FIM operations interrupted 
by the controller was not significantly more than 30% (p > 0.942). 
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Table 2. Percentage of controller-interrupted FIM operations 




Number of Controller- 

Percentage of Controller- 

Scenario 

N 

Interrupted FIM Operations 

Interrupted FIM Operations 

Wind 01/10 

8 

0 

0.0% 

Wind 06/12 

8 

1 

12.5% 


3.3 FIM Spacing Goal Conformance (MOP 3.4.1) 

Descriptive statistics associated with the spacing error at the FAF for flights with interrupted FIM 
operations (see Sections 3.2 and 3.4), flights with uninterrupted FIM operations, and all flights 
combined are shown in Table 3 and histograms are shown in Figures 13, 14, and 15. The observed 
spacing error was within ±10 sec for all flights during the experiment, with the largest errors 
occurring during the wind 06/12 scenario. 

In order to assess the following a priori hypothesis, 

Hypothesis 2: The mean spacing error at the FAF will be within ±5 seconds (sec) with a 
standard deviation of 5 sec, 

statistical analysis was performed to test the mean and standard deviation of the spacing error using 
the one-sample t-test and one-sample variance test, respectively [33], For the wind 01/10 scenario, 
the spacing error at the FAF had a mean within ±5 sec (p < 0.001) and a standard deviation 
significantly less than 5 sec (p = 0.008). 

For the wind 06/12 scenario, the two flights with interrupted FIM operations resulted in the aircraft 
crossing the FAF 2.6 sec early and 9.7 sec late. Flights with uninterrupted FIM operations had a 
mean spacing error significantly less than 5 sec (p = 0.001), but -5 sec was not a lower bound {p = 
0.304). The results are the same when all flights are combined with 5 sec being an upper bound on 
the mean spacing error (p = 0.004) but -5 sec not a lower bound (p = 0.1 09). The standard deviation 
of the spacing error at the FAF was not significantly less than 5 sec when considering either all 
FIM operations (p = 0.761) or uninterrupted FIM operations only (p = 0.234). This indicates that 
flights performing FIM operations tended to arrive early and the variability in the arrival times was 
slightly higher than desired. 
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Table 3. Descriptive statistics for spacing error at the FAF (sec) 


Scenario 

FIM 

Operations 

N 

Mean 

SD 

Min 

Median 

Max 

95% Cl on 
Mean 

95% UB 
for SD 

Wind 01/10 

All 

8 

-1.06 

2.03 

-5.0 

-0.8 

2.1 

(-2.76, 0.63) 

3.64 


All 

8 

-2.26 

5.73 

-9.1 

-2.2 

9.7 

(-7.06, 2.53) 

10.30 

Wind 06/12 

Uninterrupted 

6 

-4.20 

3.59 

-9.1 

-3.1 

-0.6 

(-7.96, -0.44) 

7.49 


Interrupted 

2 

3.55 

- 

-2.6 

3.6 

9.7 

- 

- 


Spacing Error at the FAF - All Flights 

Wind 01/10/11 



Spacing Error (sec) 


Figure 13. Spacing error at the FAF for all 
flights during the wind 01/10 scenario 


Spacing Error at the FAF - Uninterrupted FIM Operations 

Wind 06/12/11 



Spacing Error (sec) 

Figure 14. Spacing error at the FAF for 
uninterrupted FIM operations during the 
wind 06/12 scenario 


Spacing Error at the FAF - All Flights 

Wind 06/12/11 



Figure 15. Spacing error at the FAF for all 
flights during the wind 06/12 scenario 
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3.4 Percentage of Flight Deck-Interrupted FIM Operations (MOP 3.4.3) 

The number of flight deck-interrupted FIM operations was determined using the audio and video 
recordings, as well as researcher observations. A FIM operation was considered to be interrupted 
if it was terminated or suspended without being resumed more than 0.5 NM prior to the FAF. The 
percentage of flight deck-interrupted FIM operations was then computed for each scenario by 

Percentage of flight deck-interrupted FIM operations 

= 100% * number of flight deck-interrupted FIM ops / total number of FIM ops. 

One flight deck-interrupted FIM operation occurred during the first run of the wind 06/12 scenario 
(see Table 4). When the flight crew in NASA05 heard the final controller slow their target aircraft 
to final approach speed, they estimated that they were 2.5 NM behind the target. They felt 
uncomfortable with this spacing and with the commanded speeds at this time, particularly with the 
target aircraft being slowed down. The flight crew then decided to suspend FIM operations 
approximately 3.2 NM prior to the FAF. They contacted the final controller and followed the 
suspend procedure. 

In order to assess the following a priori hypothesis, 

Hypothesis 3: The rate of early FIM termination by the flight deck is less than 30%, 

statistical analysis was performed using the binomial test of one proportion to test whether the 
percentage of flight deck-interrupted FIM operations was less than or equal to 30% vs. the 
percentage was more than 30%. For both scenarios, the percentage of FIM operations interrupted 
by the flight deck was not significantly more than 30% (p > 0.942). 


Table 4. Percentage of flight deck-interrupted FIM operations 




Number of Flight Deck- 

Percentage of Flight Deck- 

Scenario 

N 

Interrupted FIM Operations 

Interrupted FIM Operations 

Wind 01/10 

8 

0 

0.0% 

Wind 06/12 

8 

1 

12.5% 


3.5 Flight Crew Acceptability of FIM Operations (MOP 3.4.4) 

The flight crew acceptability of FIM operations was measured using a 7-point Likert rating scale, 
and data were collected from each of the subject pilots via electronic post-run questionnaires. In 
order to assess the following a priori hypothesis, 

Hypothesis 4: Pilots will report the mean acceptability of FIM operations greater than or 
equal to ‘5’ (i.e., 70% ofthe 7-point scale where arating of ‘5’ indicates “Slightly Agree”), 
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data from the following items of the pilot post-run questionnaires were used. 


Item 

13. 


Item 

15 

(b). 

Item 

15 

(c). 

Item 

15 

(d). 

Item 

15 

(e). 

Item 

15 

(g). 

Item 

15 

(h). 

Item 

15 

(i)- 

Item 

15 

©• 

Item 

15 

(1). 


Please rate the overall FIM acceptability during the scenario you just 
completed. 

I was not rushed or hurried in completing the task. 

The IM commanded speeds were operationally acceptable. 

The IM commanded speeds were operationally appropriate. 

The frequency of the IM speed commands was acceptable at all times 
throughout the scenario. 

The use of voice communications to provide the IM clearance(s) was 
acceptable in this scenario. 

The amount of head down time required to input information from the IM 
clearance(s) into the EFB was acceptable. 

During this scenario, entering IM clearance information into the EFB was 
easy and intuitive. 

During this scenario, it was easy to obtain needed information from the IM 
displays. 

The flight crew procedures for the events in this scenario were acceptable. 


Data from the PF and PM were combined, and statistical analysis was performed using the 
Wilcoxon signed rank test, a nonparametric test appropriate for analyzing ordinal data with a 
within-subject design [32], 


Questionnaire item 13 used a 7-point scale with anchor points ranging from ‘1’ = “Completely 
Unacceptable” to ‘7’ = “Completely Acceptable.” Descriptive statistics associated with the pilot 
ratings of overall FIM acceptability are shown in Table 5, and histograms are shown in Figures 16 
and 17. For both scenarios, the flight crews found the FIM operations to be acceptable ( p < 0.0005). 
One pilot provided an overall FIM acceptability rating of ‘3’ and reported inadequate separation 
with the target aircraft on final despite early configuration and manual speed reductions prior to 
the FAF. This flight corresponds to the separation violation in Run 1 of the wind 06/12 scenario 
with NASA06 shown in Table 1. The pilot who rated the acceptability as ‘4’ in the wind 01/10 
scenario reported an issue with no Traffic Collision Avoidance System (TCAS) display prior to 
top of descent which caused a distraction, and indicated that the number of speed changes was 
manageable but felt they increased his workload. This same pilot also reported an overall 
acceptability rating of ‘2’ during a wind 06/12 scenario, but did not provide any additional details. 


Table 5. Descriptive statistics for ratings of overall FIM acceptability from questionnaire item 13 


Scenario 

N 

Mean 

SD 

Min 

Median 

Max 

Wind 01/10 

16 

6.375 

0.806 

4 

6.5 

7 

Wind 06/12 

16 

5.938 

1.526 

2 

6.5 

7 
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Pilot Ratings of Overall FIM Acceptability 

Wind = 01/10/11 



Pilot Ratings from Post-Run Questionnaire Item 13 


Pilot Ratings of Overall FIM Acceptability 

Wind = 06/12/11 



Figure 16. Pilot ratings of overall FIM 
acceptability during the wind 01/10 scenario 


Figure 17. Pilot ratings of overall FIM 
acceptability during the wind 06/12 scenario 


Questionnaire items 15 (b), (c), (d), (e), (g), (h), (i), (j), and (1) used a 7-point scale with anchor 
points ranging from ‘1’ = “Completely Disagree” to ‘7’ = “Completely Agree.” Descriptive 
statistics associated with the pilots’ ratings are shown in Tables 6-14, and histograms are shown 
in Figures 18-35. 

For questionnaire item 15 (b), the flight crews found the pace of the operational task to be 
acceptable for both scenarios (p < 0.0005). The rating of ‘2’ for the pace of the operational task 
was provided by the pilot who experienced the TCAS issue mentioned previously. 


Table 6. Descriptive statistics for pilot acceptability ratings of FIM operations (i.e., pace of 
operational task) from questionnaire item 1 5 (b) 


Scenario 

N 

Mean 

SD 

Min 

Median 

Max 

Wind 01/10 

16 

6.125 

1.310 

2 

6.5 

7 

Wind 06/12 

16 

6.250 

0.856 

4 

6 

7 
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Pilot Acceptability of FIM Operations - Pace of Operational Task 

Wind = 01/10/2011 



Figure 18. Pilot acceptability of FIM 
operations (i.e., pace of operational task) 
during the wind 01/10 scenario 


Pilot Acceptability of FIM Operations - Pace of Operational Task 

Wind = 06/12/2011 



Figure 19. Pilot acceptability of FIM 
operations (i.e., pace of operational task) 
during the wind 06/12 scenario 


For questionnaire item 15 (c), the flight crews found the IM commanded speeds to be operationally 
acceptable (p < 0.0005) for both wind conditions. 


Table 7. Descriptive statistics for pilot ratings of IM commanded speeds' operational acceptability 
from questionnaire item 15 (c) 


Scenario 

N 

Mean 

SD 

Min 

Median 

Max 

Wind 01/10 

16 

6.125 

0.885 

4 

6 

7 

Wind 06/12 

16 

5.875 

1.204 

4 

6 

7 


Operational Acceptability of IM Commanded Speeds 

Wind = 01/10/2011 



Pilot Ratings from Post-Run Questionnaire Item 15 (c) 


Operational Acceptability of IM Commanded Speeds 

Wind = 06/12/2011 



Pilot Ratings from Post-Run Questionnaire Item 15 (c) 


Figure 20. Operational acceptability of IM 
commanded speeds as perceived by pilots 
during the wind 01/10 scenario 
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Figure 21. Operational acceptability of IM 
commanded speeds as perceived by pilots 
during the wind 06/12 scenario 


For questionnaire item 15 (d), the flight crews found the IM commanded speeds to be operationally 
appropriate for both scenarios {p < 0.0005). The operational appropriateness rating of ‘2’ 
corresponds to the pilot who provided an overall FIM acceptability rating of ‘3’ described above 
due to a separation violation during Run 1 of the wind 06/12 scenario. 


Table 8. Descriptive statistics for pilot ratings of IM commanded speeds' operational 
appropriateness from questionnaire item 15 (d) 


Scenario 

N 

Mean 

SD 

Min 

Median 

Max 

Wind 01/10 

16 

6.125 

0.885 

4 

6 

7 

Wind 06/12 

16 

5.750 

1.483 

2 

6 

7 


Operational Appropriateness of IM Commanded Speeds 

Wind = 01/10/2011 



Pilot Ratings from Post-Run Questionnaire Item 15 (d) 


Figure 22. Pilot perceptions regarding the 
operational appropriateness of IM commanded 
speeds during the wind 01/10 scenario 


Operational Appropriateness of IM Commanded Speeds 

Wind = 06/12/2011 



Figure 23. Pilot perceptions regarding the 
operational appropriateness of IM 
commanded speeds during the wind 06/12 
scenario 


For questionnaire item 15 (e), the flight crews found the IM commanded speed frequency to be 
acceptable for both scenarios (p < 0.0005). 


Table 9. Descriptive statistics for pilot perceptions regarding the acceptability of IM commanded 
speed frequency from questionnaire item 15 (e) 


Scenario 

N 

Mean 

SD 

Min 

Median 

Max 

Wind 01/10 

16 

6.063 

1.063 

4 

6 

7 

Wind 06/12 

16 

5.813 

0.981 

4 

6 

7 
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Acceptability of IM Commanded Speed Frequency 

Wind = 01/10/2011 



Figure 24. Pilot perceptions regarding the 
acceptability of IM commanded speed 
frequency during the wind 01/10 scenario 


Acceptability of IM Commanded Speed Frequency 

Wind = 06/12/2011 



Pilot Ratings from Post-Run Questionnaire Item 15 (e) 


Figure 25. Pilot perceptions regarding the 
acceptability of IM commanded speed 
frequency during the wind 06/12 scenario 


For questionnaire item 15 (g), the flight crews found the use of voice communications to provide 
IM clearance(s) to be acceptable for both the wind 01/10 and 06/12 scenarios (p < 0.0005). 


Table 10. Descriptive statistics for pilot perceptions regarding the use of voice communications to 
provide IM clearance(s) from questionnaire item 1 5 (g) 


Scenario 

N 

Mean 

SD 

Min 

Median 

Max 

Wind 01/10 

16 

6.500 

0.632 

5 

7 

7 

Wind 06/12 

16 

6.375 

0.719 

5 

6.5 

7 


Acceptability of Voice Communications to Provide IM Clearance(s) Acceptability of Voice Communications to Provide IM Clearance(s) 

Wind = 01/10/2011 Wind = 06/12/2011 



Figure 26. Pilot perceptions regarding the use 
of voice communications to provide IM 
clearance(s) during the wind 01/10 scenario 
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Figure 27. Pilot perceptions regarding the use 
of voice communications to provide IM 
clearance(s) during the wind 06/12 scenario 


For questionnaire item 15 (h), the flight crews found the head down time required by EFB 
interactions to be acceptable for both scenarios (p < 0.0005). 

Table 11. Descriptive statistics for the acceptability of the head down time required by EFB 
interaction from questionnaire item 1 5 (h) 


Scenario 

N 

Mean 

SD 

Min 

Median 

Max 

Wind 01/10 

16 

6.375 

0.619 

5 

6 

7 

Wind 06/12 

16 

6.438 

0.727 

5 

7 

7 


Acceptability of Head Down Time associated with EFB Use 

Wind = 01/10/2011 



Figure 28. Acceptability of head down time 
required by EFB interactions during the wind 
01/10 scenario 


Acceptability of Head Down Time associated with EFB Use 

Wind = 06/12/2011 



Figure 29. Acceptability of head down time 
required by EFB interactions during the wind 
06/12 scenario 


For questionnaire item 15 (i), the flight crews found that entering IM clearance information into 
the EFB was easy and intuitive for both wind conditions (p < 0.0005). The pilot who provided a 
data entry intuitiveness rating of ‘2’ indicated he had an issue entering the target aircraft call sign 
into the EFB. 


Table 12. Descriptive statistics for the acceptability of entering IM clearance information into the 
EFB from questionnaire item 15 (i) 


Scenario 

N 

Mean 

SD 

Min 

Median 

Max 

Wind 01/10 

16 

6.500 

0.632 

5 

7 

7 

Wind 06/12 

16 

6.313 

1.401 

2 

7 

7 
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Acceptability of Entering IM Clearance Information into the EFB 

Wind = 01/10/2011 



Figure 30. Acceptability of entering IM 
clearance information into the EFB during the 
wind 01/10 scenario 


Acceptability of Entering IM Clearance Information into the EFB 

Wind = 06/12/2011 



Figure 31. Acceptability of entering IM 
clearance information into the EFB during the 
wind 06/12 scenario 


For questionnaire item 15 (j), the flight crews found that obtaining information from IM displays 
was easy for both scenarios (p < 0.0005). 


Table 13. Descriptive statistics for pilot perceptions regarding the ease of obtaining information 
from IM displays from questionnaire item 15 (j) 


Scenario 

N 

Mean 

SD 

Min 

Median 

Max 

Wind 01/10 

16 

6.438 

0.727 

5 

7 

7 

Wind 06/12 

16 

6.625 

0.619 

5 

7 

7 


Ease of Obtaining Information from IM Displays 

Wind = 01/10/2011 


Ease of Obtaining Information from IM Displays 

Wind = 06/12/2011 



Figure 32. Pilot perceptions regarding the 
ease of obtaining information from IM 
displays during the wind 01/10 scenario 
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Figure 33. Pilot perceptions regarding the 
ease of obtaining information from IM 
displays during the wind 06/12 scenario 


For questionnaire item 15 (1), the flight crews found the FIM procedures to be acceptable for both 
scenarios {p < 0.001). 


Table 14. Descriptive statistics for pilot perceptions regarding the acceptability of the flight crew 
procedures from questionnaire item 15 (1) 


Scenario 

N 

Mean 

SD 

Min 

Median 

Max 

Wind 01/10 

16 

6.438 

0.727 

5 

7 

7 

Wind 06/12 

16 

6.188 

0.834 

5 

6 

7 


Acceptability of Flight Crew Procedures 

Wind = 01/10/2011 



Pilot Ratings from Post-Run Questionnaire Item 15 (I) 

Figure 34. Acceptability of flight crew 
procedures used during the wind 01/10 
scenario 


Acceptability of Flight Crew Procedures 

Wind = 06/12/2011 



Pilot Ratings from Post-Run Questionnaire Item 15 (I) 

Figure 35. Acceptability of flight crew 
procedures used during the wind 06/12 
scenario 


3.6 Flight Crew Workload of FIM Operations (MOP 3.4.5) 

In order to assess the flight crew workload of FIM operations, the Modified Cooper-Harper (MCH) 
subjective workload rating scale was used [34], Data were collected from each of the subject pilots 
via electronic post-run questionnaires in order to assess the following a priori hypothesis, 

Hypothesis 5: Pilots will report the mean MCH workload ratings with FIM operations less 
than or equal to “3.” 

A rating of “3” on the MCH rating scale indicates that the instructed task is fair and/or has mild 
difficulty, and acceptable operator mental effort is required to attain adequate system performance. 
Descriptive statistics associated with the pilot workload ratings are shown in Table 15 and 
histograms of the data are shown in Figures 36 and 37. Data from the PF and PM were combined, 
and statistical analysis was performed using the Wilcoxon signed rank test. For both scenarios, the 
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flight crews found the workload level experienced during FIM operations to be acceptable (p < 
0 . 002 ). 


Table 15. Descriptive statistics for flight crew workload ratings 


Scenario 

N 

Mean 

SD 

Min 

Median 

Max 

Wind 01/10 

16 

1.81 

0.75 

1 

2 

3 

Wind 06/12 

16 

1.94 

0.85 

1 

2 

3 


Flight Crew Workload (ATD-1 MOP 3.4.5) 

Wind 01/10/11 


Flight Crew Workload (ATD-1 MOP 3.4.5) 

Wind 06/12/11 




Pilot Workload Ratings from Post-Run Questionnaire Pilot Workload Ratings from Post-Run Questionnaire 


Figure 36. Flight crew workload for the 
wind 01/10 scenario 


Figure 37. Flight crew workload for the 
wind 06/12 scenario 


4 Conclusions 

NASA has developed an integrated set of scheduling and spacing technologies, consisting of 
TMA-TM, CMS decision support tools, and FIM avionics and procedures. Previous studies have 
identified a number of challenges associated with the integration of these ground-based and 
airborne technologies. The ASTAR spacing algorithm was recently modified to address some of 
these air-ground challenges. 

The RAPTOR experiment described in this document was designed to assess the performance of 
the modified ASTAR spacing algorithm. ASTAR- 12 was previously evaluated in a batch study 
using deterministic scripted scenarios based on assumptions made without realistic data involving 
human participation. The RAPTOR experiment provided the first opportunity to integrate 
ASTAR- 12 with the ground-based tools. This simulation utilized an unscripted environment, and 
some of the initial assumptions made regarding the compatibility between the FIM and ground- 
based tools were found to be in need of revision. For instance, during the batch study it was 
assumed that target aircraft would fly within 10 knots of their expected published speed while on 
final approach because all allocated delay was supposed to be absorbed prior to the final approach. 
However, there were a number of cases in RAPTOR where the speed deviation on final approach 
was greater than 10 knots. 
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Due to the design of ASTAR-12, the behavior of the FIM aircraft is highly coupled to the behavior 
of the target aircraft. If a target aircraft is slowed by ATC, the FIM aircraft is also expected to slow 
down. Depending on the point where these decelerations occur on the arrival, they can cause speed 
reversals and a high frequency of low magnitude speed changes, both of which are inefficient and 
undesirable. 

Overall, the pilots reported the FIM concept, procedures, operations, and crew interface to be 
acceptable. However, the data indicate that there were some issues with the frequency of speed 
changes, speed reversals, and lower than expected speeds issued to the target aircraft by ATC prior 
to the FAF. Some of the unacceptable ratings from the pilot participants may have been influenced 
by simulation artifacts, including display of the target aircraft on the runway causing pilots to 
perceive a separation violation, and unrealistic deceleration rates for some aircraft. Inaccurate 
temperature modeling may also have affected the spacing, possibly resulting in incorrect spacing 
intervals. 
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Appendix A: Simulated Airspace and Scenario Design 


The Research and Procedural Testing of Routes (RAPTOR) experiment scenarios consisted of 85 
arrival aircraft which initialized at various points within Albuquerque Center (ZAB) and Phoenix 
Sky Harbor (PHX) Terminal Radar Approach Control (TRACON) airspace. All inbound aircraft 
approached the PHX airport via the EAGUL FIVE and MAIER FIVE standard terminal arrival 
routes (STARs) and landed on runway 26 (see Figure A-l). 



Figure A-l. Simulation airspace 
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The primary focus of the experiment was on four pairs of aircraft conducting Flight deck-based 
Interval Management (FIM) operations. Each pair consisted of a target and a following aircraft 
that used the FIM tools and procedures to achieve an assigned spacing goal at the final approach 
fix (FAF). The four FIM aircraft in the experiment were flown by subject pilots utilizing the 
Integration Flight Deck, the Development and Test Simulator, and two dual-crew Aircraft 
Simulation for Traffic Operations Research (ASTORs). The four target aircraft were flown by four 
recently retired commercial airline pilots utilizing four single crew ASTORs. Five pseudo-pilots 
operated Multi-Aircraft Control System (MACS) pseudo-pilot stations to control the 77 traffic 
arrival aircraft. 

A.l Training Scenarios 

Prior to arriving at NASA Langley Research Center, the subject pilots received a minimum of 30 
minutes of computer-based training, in addition to a Pilot’s Users Guide on FIM operations. After 
their arrival, the pilots attended a two-hour training class to familiarize them with the FIM concept 
of operations, terminology used for voice communications, and operation of the related equipment. 
Upon completion of the classroom training, the pilots flew five training scenarios with NASA 
instructors. The first four scenarios included only the FIM and target aircraft without active air 
traffic controllers or MACS pseudo-pilots. The fifth run was a fully integrated scenario with all 
participants in the loop. 

A.1.1 Training Scenario #1 

The first training scenario was a nominal run to provide practice in basic operational procedures. 
Both the target and the FIM aircraft flew the same arrival but started on different transitions (see 
Figure A-2). The target was within Automatic Dependent Surveillance-Broadcast (ADS-B) range 
at the start of the scenario and remained on path and at a stable speed for the duration of the run. 


BLD 
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GUP, 


KPHX 


Figure A-2. Training scenario #1 
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A.1.2 Training Scenario #2 


For the second training scenario, the target aircraft flew the EAGUL arrival and the FIM aircraft 
flew the MAIER arrival, as shown in Figure A-3. The target aircraft was positioned such that it 
remained outside of ADS-B range for approximately eight minutes. For simulation purposes, the 
maximum ADS-B reception range was assumed to be 120 NM with perfect reception at lesser 
ranges. After activating the Interval Management (IM) clearance, the crew received a message 
stating “IM Target No ADS-B” until the target closed to within 120 NM. Once this point was 
reached, the crew began to receive speed commands from the FIM algorithm. 


BLD 



Figure A-3. Training scenario #2 
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A. 1.3 Training Scenario #3 


For the third training scenario, both aircraft started in trail on the EAGUL arrival at flight level 
(FL) 360 (see Figure A-4). The FIM aircraft was positioned too close to the target, requiring Air 
Traffic Control (ATC) to vector the aircraft for sequencing. Prior to reaching SLIDR, the FIM 
aircraft was instructed to turn 30 degrees right, descend to FL240, and slow to 270 knots to attain 
spacing behind the preceding aircraft. After the FIM aircraft completed the turn, the instructor 
issued an IM clearance. Since the aircraft was no longer on the planned arrival, the FIM algorithm 
displayed an “Ownship off path” error message to the flight crew. Next, the crew was instructed 
to proceed direct to PAYSO to demonstrate the algorithm’s Direct-To logic. When the algorithm 
was engaged, proceeding from a location off of the arrival direct to any point on the arrival caused 
the airborne spacing algorithm to create a new arrival path and allow immediate paring with the 
target. 



Figure A-4. Training scenario #3 
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A. 1.4 Training Scenario #4 


For the fourth training scenario, both aircraft initialized in trail on the EAGUL arrival at FL360, 
as shown in Figure A-5. The instructor issued an IM clearance for NASA02 to follow 140 seconds 
behind NASA08. After the aircraft began FIM operations, the target aircraft was instructed to turn 
20 degrees right for sequencing. When the target deviated more than 2.5 NM off of the planned 
arrival path, the FIM algorithm disengaged and displayed a “Target off path” error message to the 
flight crew. At this point the target aircraft was instructed to proceed from its present position 
direct to EAGUL to demonstrate that the airborne spacing algorithm would re-engage once the 
target was again positioned within 2.5 NM of the planned arrival path. A revised spacing goal of 
110 seconds was also issued to the FIM crew to allow them to practice clearance amendments. 



Figure A-5. Training scenario #4 
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A.1.5 Training Scenario #5 


The last training scenario was a fully integrated run with the subject pilots, pseudo-pilots, and air 
traffic controllers in the loop. This run was the same as the RAPTOR data collection wind 06/12 
scenario except that the starting positions for the FIM aircraft were scrambled so that the crews 
flew a different arrival with a different target than during the data collection run. Starting positions 
for the subject aircraft are shown in Figure A-6 with the target and FIM aircraft pairs indicated by 
matching colors on the call sign labels. 


£ NASA01 
FIM 



Figure A-6. Training scenario #5 initial positions and FIM pairs 
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A.2 Data Collection Scenarios 


During planning for the RAPTOR experiment, several different scenarios were designed and 
tested. Two scenarios were selected for the final data collection runs based on the West flow traffic 
recorded at PHX on 11/14/2011 at 18:00 with the 01/10/2011 and 06/12/2011 winds. The FIM 
pairings were chosen by running a MACS scenario generated from the live traffic recording to 
observe how Traffic Management Advisor with Terminal Metering (TMA-TM) scheduled the 
arrival sequence. The FIM pairs were then selected from the arrival stream to provide a sampling 
of different target/following aircraft types with starting positions within a few minutes of the 
TMA-TM freeze horizon. The selected pairs were converted from MACS aircraft to ASTORs to 
take advantage of the ASTOR’s higher fidelity flight model and provide human-piloted targets for 
the four subject FIM aircraft. 

A.2.1 Wind 01/10 Scenario 

The RAPTOR wind 01/10 scenario included very strong wind conditions at altitude and light to 
moderate wind at the airport. TMA-TM scheduling delays for the target/FIM aircraft ranged from 
two to nearly five minutes. Starting positions for the subject aircraft are shown in Figure A-7. The 
target and FIM aircraft for each pair are indicated by matching colors on the call sign labels in the 
diagram. 



Figure A-7. Wind 01/10 scenario initial positions and FIM pairs 
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A.2.2 Wind 06/12 Scenario 


The RAPTOR wind 06/12 scenario included moderate winds at altitude with TMA-TM schedule 
delays ranging from 30 seconds to three minutes. Starting positions for the subject aircraft are 
shown in Figure A-8. The target and FIM aircraft for each pair are indicated by matching colors 
on the call sign labels in the diagram. 


^ NASA05 
FIM 


NASA71 

FIM 


BLD 



Figure A-8. Wind 06/12 scenario initial positions and FIM pairs 
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Appendix B: Experiment Schedule and Run Order 


Each two-person crew (with both members employed by the same airline) participated in a two- 
day experiment session. Every crew flew each scenario twice - once with the captain as the pilot 
flying (PF) and the first officer as the pilot monitoring (PM), and once with the first officer as the 
PF and the captain as the PM. Therefore, each crew flew a total of four experiment runs. Since 
each pilot flew each scenario, this was a within-subject experiment design. Counterbalancing was 
used to determine the run order of the scenarios, and the pilots switched responsibilities between 
runs, resulting in the run order shown in Table B-l. 


Table B-l. Experiment run order 


Run 

Scenario 

PF and PM 

Run 1 

Wind 06/12 

Captain as PF and First Officer as PM 

Run 2 

Wind 01/10 

First Officer as PF and Captain as PM 

Run 3 

Wind 01/10 

Captain as PF and First Officer as PM 

Run 4 

Wind 06/12 

First Officer as PF and Captain as PM 


One group of five recently retired air traffic controllers served as confederate controllers actively 
controlling the aircraft. This included two Center controllers (ZAB Sectors 43 and 39), one PHX 
Feeder controller, one PHX Final controller, and one Ghost controller. All controllers were trained 
at NASA Ames Research Center in the use of the ground-based tools and had participated in 
previous ATD-1 experiments. The Center and Ghost controllers rotated positions between runs as 
shown in Table B-2. 


Table B-2. Rotation of controllers 


Run 

Ghost 

Center Sector 43 

Center Sector 39 

Feeder 

Final 

Run 1 

A 

B 

C 

D 

E 

Run 2 

B 

C 

A 

D 

E 

Run 3 

C 

A 

B 

D 

E 

Run 4 

A 

B 

C 

D 

E 


Each pilot participated in a two-day experiment session with the schedule shown in Figure B-l. 
The pilot participants received training material and access to computer based training prior to 
arriving at NASA Langley Research Center. They also received approximately six hours of 
classroom and hands-on training after arrival, including flying five training scenarios prior to 
commencing data collection. The first day of the experiment session began with classroom and 
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hands-on training, and then one data collection flight was conducted. The second day consisted of 
the remaining data collection flights, followed by a post-experiment questionnaire and group 
debrief session. Pilots also completed post-run experiment questionnaires following each data 
collection run. Post-run and post-experiment questionnaires were administered to and feedback 
was collected from the confederate controllers. 


Figure B-l. Daily experiment schedule 


Day #1 


Day #2 

Intro Brief 


Data Collection Run #2 

Classroom Training 


Data Collection Run #3 

Training Scenarios 


Lunch 

Lunch 


Data Collection Run #4 

Training Scenarios 


Post-Experiment Questionnaire 

Data Collection Run #1 


Debrief 
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Appendix C: FIM Crew Interface and Procedures 


C.l Overview of Interface and Procedures 

The prototype Flight deck-based Interval Management (FIM) crew interface consists of two side- 
mounted electronic flight bags (EFB) and configurable graphics display (CGD) devices mounted 
on the left and right outboard panels in the pilot’s forward field-of-view. Pilots use the EFB for 
data entry, as well as to activate, amend, suspend, resume, and cancel the FIM operation, while the 
CGD is used for conformance monitoring. 

The CGD displays five pieces of information in the pilot’s forward field-of-view (see Figure C-l). 


1) CMD SPD displays the Interval 
Management (IM) commanded speed 
in Mach or knots. When an IM speed 
change occurs, the speed is shown in 
reverse video for 10 seconds. If the 
pilot does not respond within 10 
seconds, the commanded speed blinks 
until the speed is set in the Mode 
Control Panel (MCP). 

2) The status box displays the current 
status of the FIM operation: 

CALCULATING, SPACING, or 
SUSPENDED. 

3) The target aircraft call sign is 
displayed when receiving Automatic 
Dependent Surveillance - Broadcast 
(ADS-B) information. 

4) The Fast / Slow indicator displays the difference between the actual and instantaneous 
Interval Management (IM) commanded airspeed. During deceleration this value may be 
used to see how closely the FIM aircraft is to the desired deceleration rate. 

5) System caution and information messages are displayed in the message box. 


f © 


CMD SPD O 

260 KT 


DRAG RED 


SPAC ING © 
UAL 234 ® 


Figure C-l. Information elements displayed 
on the CGD 
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Figure C-2. Information elements displayed on the EFB 
The EFB enables data entry and displays the information shown in Figure C-2. 

1) CMD SPD displays the current IM commanded speed the aircraft should fly in order to 
achieve the spacing goal. 

2) The Fast / Slow indicator displays speed conformance information. The deviation of the 
actual speed from the instantaneous commanded speed is shown. 

3) The status box displays the current status of the IM operation, including CALCULATING, 
SPACING, and SUSPENDED. 

4) The message box displays caution and information messages generated by the FIM system. 

5) The Situation Display shows the location of the target and other ADS-B out aircraft. The 
ownship aircraft is shown as a white triangle centered in the display. The target aircraft is 
shown as two chevrons with the inner chevron white and the outer green. The target 
aircraft’s information block is also displayed with call sign and altitude information. Other 
ADS-B out aircraft are shown as blue chevrons. 
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6) OWN INFO allows the pilot to enter ownship information into the FIM system. 

7) Next Waypoint displays the next waypoint on the arrival that the aircraft will fly over. This 
field is for information only and cannot be modified by the pilot. 

8) Descent Forecast Winds allows the pilot to enter up to eight altitude winds for the arrival. 
This information may be entered manually or uploaded with an Aircraft Communications 
Addressing and Reporting System (ACARS) or Controller-Pilot Data Link 
Communications (CPDLC) message. 

9) IM GOAL allows the pilot to enter the assigned spacing goal. 

10) TGT ACFT allows the pilot to select and identify the target aircraft to space behind. 

11) TGT RTE allows the pilot to enter the target aircraft’s arrival routing, transition, and 
approach so that the FIM system knows the target’s ground track. 

12) Using the ZOOM IN and ZOOM OUT buttons changes the range on the EFB’s situation 
display. 

13) The bezel buttons on the top of the EFB: 

• MENU displays the EFB software’s Main Menu 

• BACK returns to previous main page 

• PGUP and PGDN cycles through pages if more than one is needed to display 
information 

• XFR is inoperative 

• ENTER button enters inputted information into the FIM system 


The FIM system provides commanded speeds for the pilot to fly in order to achieve a precise 
interval behind the target aircraft to the final approach fix (FAF). Pilot actions are essentially the 
same as for current day operations. He or she flies the commanded speeds while maintaining the 
vertical profile to meet all restrictions. The main difference is that the commanded speeds come 
from the FIM avionics instead of the controller. 

If Air Traffic Control (ATC) expects the pilot to fly FIM procedures, the controller will issue an 
IM clearance. This clearance will contain the spacing interval to achieve, the target aircraft’s call 
sign, and the target aircraft’s arrival routing. An aircraft is considered paired with a target aircraft 
once valid ADS-B information from that target aircraft is received, and a commanded airspeed is 
displayed. Once paired with a target aircraft, an IM commanded speed is generated for the pilot to 
fly that will achieve the assigned spacing goal when the aircraft crosses the FAF. The system is 
designed for limited airspeed changes and to conform as closely as possible to an Optimized Profile 
Descent (OPD). At airports saturated with arrival aircraft, the greatest capacity benefits may be 
realized by having sequences of aircraft operating in IM mode, with each aircraft actively spacing 
behind the aircraft ahead of it. 
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Speed guidance is displayed on the EFB in the CMD SPD block and is duplicated on the CGD 
located in the pilot’s forward field-of-view. The aircraft’s airspeed is controlled by setting the IM 
commanded speed in the MCP speed window. The pilot flying will fly the arrival and instrument 
approach on autopilot. The use of the autopilot system reduces pilot workload and allows precise 
spacing intervals to be established. For a majority of the descent, the aircraft will descend in 
vertical navigation (VNAV) SPD mode with the throttles in the HOLD mode. To ensure the 
predictability of vertical paths during FIM operations, flight crews will be required to modulate 
thrust and drag to stay on the IM speed profile and the OPD path. The aircraft will pitch to maintain 
the speed window’s set speed. Throttles and drag devices will be used to nominally maintain the 
aircraft within ±400 feet of the VNAV path. After crossing the FAF, IM speed guidance will be 
removed from the display, and the pilot will configure the aircraft for landing. 

An overview of the procedures to perform IM operations is as follows. 

1 . Program flight management computer (FMC) with arrival routing, VNAV descent, and 
forecast winds. Tune radios. 

2. Load ownship information into EFB 

3. Load descent forecast winds into EFB 

4. Load assigned spacing goal into EFB 

5. Load target aircraft call sign into EFB 

6. Load target aircraft arrival routing into EFB 

7. Activate IM in EFB 

8. Fly IM commanded airspeed on arrival while maintaining VNAV path 

9. At FAF configure airplane for landing 

Details on loading information into the EFB, as well as the activation, suspension, resumption, 
and cancellation of FIM operations are provided below. 
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C.2 Load Ownship Information into EFB 

Ownship information is typically programmed earlier in the flight. Figures C-3 through C-8 
illustrate the procedure for loading the ownship route information into the EFB. 



Press the OWN INFO button to enter the ownship route 
information. 


Figure C-3. Load ownship information 



Enter information into the scratch pad and load into the 
fields. 

CRZ ALT: Three number value (e.g. 350) for expected 
altitude at Top of Descent (TOD) 

CRZ MACH: Two number value (e.g. 80) for expected 
Mach within 100 miles of TOD 

DES MACH/CAS: Two number slash three number value 
(e.g. 80/300) for descent Mach and crossover CAS 


Figure C-4. Load ownship cruise altitude, cruise Mach, and descent Mach/CAS transition 
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Enter the arrival airport’s ICAO identifier in the 
DEST AIRPORT block. Then the OWNSHIP RTE 
button will appear. Press the OWNSHIP RTE button 
to input the ownship’s arrival, transition, and 
approach to the airport. 


Figure C-5. Load the ownship arrival airport 



Select the arrival, transition, and approach by 
touching the screen or using the bezel buttons. The 
RESET button may be pressed to start over if a 
mistake was made. The ENTER button will appear 
when all the information is selected. Press the 
ENTER button to finish inputting the ownship route 
information and return the EFB to the OWNSHIP 
INFO page. 


Figure C-6. Load the ownship arrival, transition, and approach to the airport 
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Route now appears in the OWNSHIP RTE block. 
Once all required information is entered, an ENTER 
button appears. Press the ENTER button to input the 
information and return the EFB to the Interval 
Management Main Page. 


Figure C-7. Enter the ownship route information 



OWN INFO block displays ownship route to indicate 
that ownship information has been entered. 


Figure C-8. Interval Management Main Page with ownship route information 
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C.3 Load Descent Forecast Winds into EFB 


For FIM operations to work efficiently, accurate wind information must be available. In practice, 
this wind forecast will be provided by the aircraft’s company and will be at altitudes that have 
major wind shifts. The wind information may be uploaded or entered manually as illustrated in 
Figures C-9 through C- 1 1 . 
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Press the DES FCST WINDS block to enter the 
descent forecast winds. 


Figure C-9. Load the descent forecast winds 
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Enter the altitude into the scratchpad. Three 
number values will be interpreted as Flight Levels 
while four and five number values will be entered 
as MSL altitudes. Enter the value by pressing the 
altitude block or the adjacent bezel button. The 
BKSP button will erase one number while the 
CLR button will clear the whole scratchpad. To 
delete a value already loaded into the field, press 
the DELETE button and then the desired field. 

Enter the direction and speed values separated by 
a slash. Enter the value by pressing the associated 
DIR/SPEED block or the adjacent bezel button. 

Press the ENTER key located on the touchscreen 
or the top of the EFB to return to the EFB Interval 
Management Main Page. 


Figure C-10. Load the wind direction, speed, and altitudes 
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There are two Descent Forecast Winds Pages 
allowing a total of eight altitude winds for the 
descent to be entered. Use the PGUP and 
PGDN buttons located on the top of the EFB 
to switch between pages. 


Figure C-l 1. Switch between pages on the EFB 


C.4 Load Assigned Spacing Goal and Target Aircraft Information into EFB 

An example of an IM clearance issued by ATC over the radio is given below. 

‘NASA 12, when able space 90 seconds behind Delta Alpha Lima 877 on EAGUL 
5 Zuni Transition. Report paired. ’ 

Figures C-l 2 through C-20 illustrate the procedure for loading the assigned spacing goal, target 
aircraft call sign, and target aircraft arrival routing information provided in the clearance. 
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Press the IM GOAL button to enter the assigned 
spacing goal. 


Figure C-12. Load the assigned spacing goal 
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ENTER IM GOAL 


FORMAT: SSS OR D.DNM (SSS=SECONDS . D.D=DIST) 

EXAMPLE: 90 (GOAL IN SECONDS) 

2.5NM (GOAL IN NAUTICAL MILES) 
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Enter value in scratchpad and then press ENTER. 
IM GOAL: Two number value (e.g. 90) 


Figure C-13. Enter the assigned spacing goal 
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The IM GOAL now shows 90 seconds. To enter the 
target aircraft call sign press the TGT ACFT block 
on the touchscreen or the bezel button located 
beside it. 


Figure C-14. Load the target aircraft call sign 
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The Target Aircraft page lists all the aircraft within 
ADS-B range sorted alphabetically. If there are 
more aircraft than can fit on one page, the PGUP 
and PGDN buttons can be used. Select the target 
aircraft by either pressing on the touchscreen or 
pressing the bezel button located beside it. 

Once a call sign is selected, the ENTER and 
CLEAR buttons will be active. Pressing the 
CLEAR button will clear the selection. Pressing the 
ENTER button will enter the call sign into the FIM 
system and return to the Interval Management Main 
Page. 


Figure C-15. Select the target aircraft call sign from the list of aircraft within ADS-B range 



If the target aircraft is not in ADS-B range, the pilot 
may enter it manually by pressing the MANUAL 
button. 


Figure C-16. Manually load the target aircraft call sign 
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Enter the call sign of the target aircraft. The input 
can be cleared by using the CLR button located on 
the bottom row of the keyboard. The CANCEL 
button will cancel the input and return to the Target 
Aircraft Page. Pressing the ENTER button will 
enter the call sign into the FIM system and return to 
the Interval Management Page 


Figure C-17. Manually enter the target aircraft call sign 



The target aircraft call sign is now shown in the 
TGT ACFT block. To enter the target aircraft’s 
route press the TGT RTE block or the bezel button 
located beside it. 


Figure C-18. Load the target aircraft route information 
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Select the target aircraft’s arrival, transition, and 
approach in the same manner as the ownship route 
information was selected. 


Once all items are selected, an ENTER button 
appears. Press the ENTER button to input the 
target aircraft’s route into the FIM system and 
return to the main Interval Management Main 
Page. 


Figure C-19. Select target aircraft route information 



The target aircraft’s arrival and transition is shown 
in the TGT RTE block. 


Figure C-20 Interval Management Main Page with assigned spacing goal, target aircraft call 
sign, and target aircraft route information 
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C.5 Activate IM System 

When all the required information has been entered into the FIM system, the ACTIVATE button 
on the EFB will become selectable as shown in Figure C-21. Once the information has been 
verified by both pilots, the ACTIVATE button is pressed to begin performing FIM operations. If 
the IM clearance specified initializing spacing over a waypoint, pressing ACTIVATE is delayed 
until the aircraft is crossing that waypoint. 
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Figure C-21. Activate the IM operation 
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The FIM system will initially need time to compute speed commands. During this period, 
CALCULATING will be displayed on the EFB (see Figure C-22). The TGT ACFT and TGT RTE 
input fields turn into labels, and cannot be changed. The OWN INFO and IM GOAL fields can be 
changed if the original IM clearance is amended. The NEXT WPT field displays the next waypoint 
the ownship will fly over on the arrival. The CGD located in the pilot’s forward field-of-view will 
also become active displaying CALCULATING and the target aircraft’s call sign. The call sign is 
shown in white to indicate the target aircraft is not yet paired and no commanded speed is shown. 



Figure C-22. Calculating initial speed command 
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Once a commanded speed has been calculated, the ownship and target aircraft will be paired, and 
a commanded speed will appear as shown in Figure C-23. The pilot will then open the MCP speed 
window and fly this speed. The Fast / Slow indicator appears to provide speed conformance 
information. The CGD will also change to indicate the paired status. SPACING will appear in the 
Status block, and the target aircraft’s call sign will turn green. A commanded speed will appear 
and the Fast / Slow indicator will be presented graphically on the left side. Once paired, the pilot 
may fly the FIM operation using the CGD exclusively. 
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Figure C-23. Information displayed on the EFB and CGD while performing FIM operations 
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C.6 Arrival Interval Management Procedures 

■ (PF, PM) Airspeed Requirements 

o Observe and announce IM Speed changes and mode changes on CGD/EFB 

■ Speed changes will highlight for 10 seconds and then will blink if not set 
o Set IM commanded speed in speed window on MCP 
o Maintain ±10 knots of IM commanded speed during speed changes 

NOTE: When IM is active, fly the IM commanded speed and disregard any charted 
speeds on the arrival. Use the FAST/SLOW indicator for deceleration/acceleration rate 

guidance. 

o Configure aircraft as necessary to maintain IM commanded speed 
o Airspeed is safe and acceptable to the pilot for current conditions (See non normal 
below for action) 

■ (PF, PM) Vertical Path Requirements 

o Verify VNAV SPD is active mode 
o Ensure aircraft starts a descent at TOD Point 

o Use drag devices and thrust as necessary to maintain VNAV path within ±400 feet 
(PF) 

o Monitor that aircraft stays on path and all restrictions will be met 

■ (PF, PM) Spacing Requirements 

o No Caution messages on EFB (See non normal below for action) 
o Notify ATC when initially spacing behind target aircraft 
o Notify each new ATC check in with “Paired with” 
o Notify ATC if no longer IM spacing 
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C.7 Final Segment Interval Management Procedures 

■ (PF, PM) Configuration and Energy Management 
o Extend Flaps as necessary 
o VNAV PTH will engage at flap extension 

o Maintain least amount of flaps required to maintain IM speed and vertical path 

NOTE: IM commanded speed may increase above current flap max speed. Reduction 
of flaps may be required. 

o When IM commanded speed blanks at the FAF 

■ Gear down 

■ Target Speed set in MCP Window 

■ Configure as necessary to be stable by 1000 feet AGL 

• Automation Procedures 

o Aircraft will transition to VNAV PTH when flaps are extended 
o Arm approach mode between 6-2 miles prior to FAF 
o Ensure aircraft will capture both the localizer and glideslope 

NOTE: If aircraft is on VNAV PTH profile the aircraft will be on or slightly below 
glideslope when established on final 

o Set Target Speed in MCP speed window when crossing the FAF and IM 
Commanded Speed is removed from the EFB and CGD. 
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C.8 Amendment of Spacing Goal 

ATC may amend the IM clearance with a new assigned spacing goal. The procedure for amending 
the spacing goal in the EFB is illustrated in Figure C-24. 



ATC may amend the IM clearance with a new IM 
GOAL. 

Press IM GOAL and enter the new value into the 
system. The other pilot will check the new value after 
it has been entered. 


Figure C-24. Amend the assigned spacing goal 


C.9 Suspend and Resume 

ATC may need to suspend FIM operations for a period of time. If a Suspend Instruction, such as 

‘NASA 6, suspend interval spacing and slow 10 knots ’ 

is issued, the flight crew will assume that FIM operations will resume at a later time. ATC may 
later issue a Resume Instruction such as the example given below. 

‘NASA 6, when able, resume interval spacing with DAL877’ 

The procedure for suspending and resuming IM spacing is illustrated in Figures C-25 through C- 
28. 


62 





Upon receiving a Suspend Instruction, press the 
SUSPEND button on the Interval Management Main 
Page. 


Figure C-25. Suspend IM spacing 



All speeds are removed from the EFB and CGD and 
SUSPENDED is displayed in the Interval 
Management Main Page Status Box. Follow ATC 
instruction for airspeed. 


Figure C-26. Interval Management Main Page with IM spacing suspended 
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Upon receiving a Resume Instruction, press the 
RESUME button on the Interval Management Main 
Page. 


Figure C-27. Resume IM spacing 
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All IM airspeed information is displayed on the EFB 
and CGD. Follow the CMD SPD at this time. 


Figure C-28. Interval Management Main Page with IM spacing resumed 
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C.10 Unable Spacing 


If the CMD SPD disappears from the interface due to a caution (see Figure C-29), the flight crew 
flies the current airspeed and advises ATC. ATC will determine the appropriate action. If ATC 
instructs, “advise when able to resume spacing,” the crew will monitor the CMD SPD block. If a 
value returns to the block the crew will advise ATC and follow instructions. Once the CMD SPD 
returns, the flight crew reports to ATC that they are able to space behind the target aircraft. ATC 
may then instruct the aircraft to resume spacing. 
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Figure C-29. Unable spacing due to target off path 


65 



C.ll Cancel Spacing 


ATC may cancel the IM clearance, which will be followed by airspeed/heading instructions or a 
new IM clearance. The procedure for cancelling IM spacing using the EFB is illustrated in Figures 
C-30 and C-31. 



Upon receiving a Cancel Instruction, press the SUSPEND 
button on the Interval Management Main Page. 


Figure C-30. Cancel IM spacing by first suspending FIM operations 



Press the CANCEL IM button to cancel the IM clearance. 
This will remove all information from all fields. A new 
clearance will have to be entered to Activate Interval 
Spacing. 


Figure C-31. Cancel IM clearance 
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C.12 Alerts: Cautions, Advisory, and Memos 

The IM system and its associated ASTAR algorithm have the set of alerts shown in Table C-l. 


Table C-l. IM system alerts 


Alert 

Level 

EICAS Message 

Meaning 

Pilot Action 

Caution 

IM DISENGAGED 

Loss of ownship flight 
path data, failure of the 
interface between the 
spacing algorithm and 
the aircraft avionics, 
ADS-B receiver failure, 
or other aircraft avionic 
failures 

Fly Current Airspeed and 
advise ATC “Unable 
Spacing due equipment 
failure” 

Caution 

IM TGT OFF PATH 

Target aircraft is not on 
the flight path given by 
the ATC IM clearance 

Fly Current Airspeed and 
advise ATC “Unable 
Spacing due to Target off 
path” 

Caution 

IM TGT ADSB LOST 

Target aircraft ADS-B 
information is lost 

Fly Current Airspeed and 
advise ATC “Unable 
Spacing due to Target ADS- 
B Loss” 

Caution 

IM OWN OFF PATH 

Aircraft is greater than 
2.5 NM laterally, 6000’ 
vertically, or 90 degrees 
of heading from the 
planned flight path 

Fly Current Airspeed and 
advise ATC “Unable 
Spacing due to Ownship off 
path” 

Advisory 

IM SPD LIMITED 

IM would command a 
different speed but is 
limited by the 15% 
constraint 

Advisory only. No crew 
action. 
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Appendix D: Pilot Post-Run Questionnaire 


1. What is your name? 

**Names will be removed and replaced with coded ID numbers. 

2. Please select the scenario you just completed from the list below: 

• Training 

• Scenario 1 

• Scenario 2 

• Scenario 3 

• Scenario 4 

3. Please select your role during the scenario 

• Pilot Flying 

• Pilot Not Flying / Pilot Monitoring 

4. Please select your role during the scenario 

• Captain 

• First Officer 

5. What was your call sign during the previous run? 

• NAS AO 1 

• NASA02 

• NASA05 

• NASA06 

6. Even though errors may be large or frequent, can instructed task be accomplished most 
of the time? 

• Yes 

• No 

7. [If pilot responded ‘Yes’ to Item 6] Are errors small and inconsequential? 

• Yes 

• No 

8. [If pilot responded ‘No’ to Item 7] Given that major deficiencies exist and system redesign 
is strongly recommended, please choose one of the following ratings: 

• Major difficulty / maximum operator mental effort is required to bring errors to 
moderate level 

• Major difficulty / maximum operator mental effort is required to avoid large or 
numerous errors 

• Major difficulty / intense operator mental effort is required to accomplish task, but 
frequent or numerous errors persist 


you just completed from the list below: 


you just completed from the list below: 
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9. [If pilot responded ‘Yes’ to Item 7] Is mental workload level acceptable? 

• Yes 

• No 

10. [If pilot responded ‘No’ to Item 9] Given that mental workload is high and should be 
reduced, please choose one of the following ratings: 

• Minor but annoying difficulty / moderately high operator mental effort is required to 
attain adequate system performance 

• Moderately objectionable difficulty / high operator mental effort is required to attain 
adequate system performance 

• Very objectionable but tolerable difficulty / maximum operator mental effort is 
required to attain adequate system performance 

11. [If pilot responded ‘Yes’ to Item 9] Given that mental workload level was acceptable, 
please choose from one of the following: 

• Very easy / highly desirable / operator mental effort is minimal and desired 
performance is easily attainable 

• Easy, desirable / operator mental effort is low and desired performance is attainable 

• Fair, mild difficulty / acceptable operator mental effort is required to attain adequate 
system performance 
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Difficulty Level 


Operator Demand Level 


Rating 



Even Though 
Errors May Be 
Large Or Frequent, 
Can Instructed Task 
BeAccomplished 
^•Most Of The Time? 


Very Easy, 
Highly Desirable 

Operator Mental Effort Is Minimal And 
Desired Performance Is Easily Attainable 

jfiji 

Easy, 

Desirable 

Operator Mental Effort Is Low 

And Desired Performance Is Attainable 


Fair, 

Mild Difficulty 

Acceptable Operator Mental Effort Is 
Required To Attain Adequate System 
Performance 

irvj 




Minor But Annoying 
Difficulty 

Moderately High Operator Mental Effort Is 
Required To Attain Adequate System 
Performance 


Moderately Objectionable 
Difficulty 

High Operator Mental Effort Is Required To 
Attain Adequate System Performance 

SrTj! 

Very Objectionable But 
Tolerable Difficulty 

MaximumOperator Mental Effort Is Required 
To Attain Adequate System Performance 

j^6 




Major Difficulty 

Maximum Operator Mental Effort Is Required 
To Bring Errors to Moderate Level 

;f7ij 

Major Difficulty 

Maximum Operator Mental Effort Is Required 
To Avoid Large or Numerous Errors 

TYJi 

Major Difficulty 

Intense Operator Mental Effort Is Required 
To Accomplish Task, But Frequent Or 
Numerous Errors Persist 

jFTi 



Major 

Deficiencies 
System 
Redesign 
is Mandatory 


Impossible 


Instructed Task Cannot 
Be Accomplished Reliably 



Operator Decisions 


12. Follow the flow chart above to select the peak workload you experienced during each 
segment of flight during the scenario you just completed. Rating of your peak workload 
level: 


>1 8,000ft (cruise, initial descent) 

18.000- 11,000 (descent, approach check) 

1 1.000- 5,000 (TRACON, low altitude merge) 
<5,000 (final approach, configure aircraft) 


l 

2 

3 

4 

5 

6 

7 

8 

9 

10 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 
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13. Please rate the overall FIM acceptability during the scenario you just completed. 


Completely 

Unacceptable 

1 


2 


3 


4 5 


6 


Completely 

Acceptable 

7 


14. Please rate the FIM acceptability during each segment of flight during the scenario you 
just completed. 


Completely 


Moderately 


Very 

Unacceptable 


Acceptable 


Acceptable 


1 

2 

3 

4 

5 

>18,000 ft (cruise, initial descent) 

o 

o 

o 

o 

o 

18,000-11,000 (descent, approach check) 

o 

o 

o 

o 

o 

1 1,000-5,000 (TRACON, low altitude merge) 

o 

o 

o 

o 

o 

<5,000 ft (final approach, configure aircraft) 

o 

o 

o 

o 

o 
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15. Respond to each of the statements shown below using a scale ranging from “1” 
(Completely Disagree) to “7” (Completely Agree). 


Completely Completely 

Agree Disagree 



1 

2 

3 

4 

5 

6 

7 

Relevant information, including operational 
plans, decisions, and changes in aircraft state, 
were effectively communicated between 
yourself and your crew member. 

O 

o 

o 

o 

o 

o 

o 

I was not rushed or hurried in completing the 
task. 

O 

o 

o 

o 

o 

o 

o 

The IM commanded speeds were 
operationally acceptable. 

O 

o 

o 

o 

o 

o 

o 

The IM commanded speeds were 
operationally appropriate. 

O 

o 

o 

o 

o 

o 

o 

The frequency of the IM speed commands 
was acceptable at all times throughout the 

O 

o 

o 

o 

o 

o 

o 

scenario. 








I understood why the IM speed commands 
were provided (i.e. the IM commanded 
speeds made sense). 

o 

o 

o 

o 

o 

o 

o 

The use of voice communications to provide 
the IM clearance(s) was acceptable in this 

o 

o 

o 

o 

o 

o 

o 

scenario. 








The amount of head down time required to 
input information from the IM clearance(s) 
into the EFB was acceptable. 

o 

o 

o 

o 

o 

o 

o 

During this scenario, entering IM clearance 
information into the EFB was easy and 
intuitive. 

o 

o 

o 

o 

o 

o 

o 

During this scenario, it was easy to obtain 
needed information from the IM displays 

o 

o 

o 

o 

o 

o 

o 

The flight crew procedures for the events in 
this scenario were complete. 

o 

o 

o 

o 

o 

o 

o 

The flight crew procedures for the events in 
this scenario were acceptable. 

o 

o 

o 

o 

o 

o 

o 
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16. Did the procedures contain missing steps? 

• Yes 

• No 

17. Did the procedures contain extra steps that were unnecessary? 

• Yes 

• No 

18. Were the procedural steps logical and easy to follow? 

• Yes 

• No 

19. Please briefly explain any undesirable ratings from the statements above. 


20. Describe any unusual or unexpected event(s) and your reaction(s), if applicable. 


21. This space is reserved for any additional comments related to awareness and 
acceptability issues. If you have any clarifying comments or interesting observations 
related to awareness and acceptability issues, please provide them below. 
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Appendix E: Pilot Post-Experiment Questionnaire 


1. What is your name? 

** Names will be removed and replaced with coded ID numbers. 


2. What was your call sign during the experiment? 

• NASA01 

• NASA02 

• NASA05 

• NASA06 


3. Was the workload required to operate the simulator much less than, the same as, or 
greater than the workload required to fly an actual aircraft? 


Workload 

Required 


Much 

More 

1 


Moderately Slightly The 

More More Same 

2 3 4 


Slightly Moderately Much 
Less Less Less 

5 6 7 


4. Please share your impressions of the flight scenarios (e.g. comment on their level of 
realism, appropriateness, and/or diversity) and comment on how the design of the 
scenarios impacted your ability to perform the spacing task: 


5. Did you receive adequate training with respect to flying the simulator? 

• Yes 

• No. If not, please briefly describe how simulator training can be improved. 
Make a comment on your choice here: 
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6. Did you receive adequate training with respect to the IM spacing procedure and the 
spacing tool? 

• Yes 

• No. If not, please briefly describe how IM procedures or spacing tool training can be 
improved. 

Make a comment on your choice here: 


7. Did you receive adequate training with respect to the entry and interpretation of 
information presented on the EFB? 

• Yes 

• No. If not, please briefly describe how EFB training can be improved. 

Make a comment on your choice here: 


8. Did you receive adequate training with respect to the interpretation of information 
presented on the CGD in the forward field of view? 

• Yes 

• No. If not, please briefly describe how CGD training can be improved. 

Make a comment on your choice here: 
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Interval Management Procedures 

The following questions are intended to gather your feedback about the procedures used for each 
aspect of the IM operation that was tested (note that the last question asks about the general IM 
procedures). 

9. Were the procedures for terminating an IM operation complete, accurate, and logical? 

• Yes 

• No. Please provide any suggestions regarding the way(s) in which the procedures for 
terminating the IM operation may be improved. 

Make a comment on your choice here: 


10. Were the procedures for suspending and then resuming an IM operation complete, 
accurate, and logical? 

• Yes 

• No. Please provide any suggestions regarding the way(s) in which the procedures for 
suspending the IM operation may be improved. 

Make a comment on your choice here: 


11. Were the procedures for amending the IM spacing goal complete, accurate, and logical? 

• Yes 

• No. Please provide any suggestions regarding the way(s) in which the procedures for 
amending the IM spacing goal may be improved. 

Make a comment on your choice here: 
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12. Were the procedures for reacting to the loss of your lead aircraft’s ADS-B signal 
complete, accurate, and logical? 

• Yes 

• No. Please provide any suggestions regarding the way(s) in which the procedures for 
reacting to the loss of your lead aircraft’s ADS-B signal may be improved. 

Make a comment on your choice here: 


13. Were the general (nominal) IM procedures complete, accurate, and logical? 

• Yes 

• No. Please also provide any suggestions regarding the way(s) in which the general IM 
procedures may be improved. 

Make a comment on your choice here: 


14. Was the IM phraseology used in this experiment correct and intuitive? 

• Yes 

• No. If “no,” why not, and what could be done to improve the phraseology? 
Make a comment on your choice here: 


15. How difficult do you think it would be for a typical crew to learn and integrate the IM 
spacing procedures into their current daily operational flight procedures? 

Very Moderately Slightly . . . Slightly Moderately Very 

Difficult Difficult Difficult eU ra Easy Easy Easy 

1 2 3 4 5 6 7 
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16. Briefly describe any challenges involved with integrating the IM procedures with existing 
procedures. 


17. Given the experience with IM that you gained during this simulation, what is your overall 
assessment of the safety of the spacing procedure compared with current day operations? 
(“Safety” in this question refers to your holistic opinion to include workload, awareness, 
position relative to other aircraft, etc.) 


Not Safe Moderately Slightly . _ f 

At All Less Safe Less Safe As bate 

12 3 4 


Slightly 

More 

Safe 

5 


Moderately 
More Safe 

6 


Much 

More 

Safe 

7 


18. In general, did you find the process of entering IM clearance information into the EFB 
easy and intuitive? 

• Yes 

• No. If “no,” what can be done to improve the process of loading information into the 
EFB? 

Make a comment on your choice here: 


19. Did the CGD and EFB provide you with the information you needed/desired to safely 
and correctly conduct IM, and was this information easy to obtain when needed? 

• Yes 

• No. If “no,” what information was missing, or how can the information be presented 
better? 

Make a comment on your choice here: 
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20. Did following the IM commanded speed and procedure ever cause unexpected or 
undesired behavior? 

• Y es. If “yes,” please explain what the unexpected or undesired behavior was. 

• No 

Make a comment on your choice here: 


21. Did you find the responsibility of using onboard automation to achieve a spacing interval 
behind a lead aircraft acceptable (when ATC is responsible for separation)? 

• Yes 

• No. If “no,” why not, and what could be done to make the responsibility or workload 
acceptable? 

Make a comment on your choice here: 


22. Did you find your level of engagement with the IM automation acceptable (i.e., the level 
of decision making ability you had, and your understanding of the reasoning behind IM 
speeds that were commanded)? 

• Yes. Please explain. 

• No. Please explain. 

Make a comment on your choice here: 
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23. Did the IM commanded speeds make sense? 

• Yes 

• No 

Make a comment on your choice here: 


24. Do you have any additional comments about the experiment? 
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Appendix F: Controller Post-Run Questionnaire 


1. What is your name? 

** Names will be removed and replaced with coded ID numbers. 


2. Please select the scenario you just completed from the list below: 

• Training 

• Scenario 1 

• Scenario 2 

• Scenario 3 

• Scenario 4 

3. Which position did you work in this scenario? 

• Center North (ZAB 36/43) 

• Center North (ZAB 93/39) 

• Feeder North 

• Final North 

• Ghost 

4. How mentally demanding was the task? 

Very 

Low 

1 2 3 4 5 6 

5. How hurried or rushed was the pace of the task? 

Very 

Low 

1 2 3 4 5 6 


Very 

High 

7 


Very 

High 

7 


6. How hard did you have to work to accomplish your level of performance? 


Very 

Low 

1 2 3 4 5 6 


Very 

High 

7 


7. How insecure, discourage, irritated, stressed, and annoyed were you? 


Very 

Low 

1 


4 


Very 

High 

7 


2 


3 
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5 


6 


8. How physically demanding was the task? 


Very 

Low 

1 


2 


3 


4 


5 


6 


Very 

High 

7 


9. How successful were you in accomplishing what you were asked to do? 


Good 


Poor 

7 


2 


3 


4 


5 


6 


10. Were the ATD-1 operations in this last run safe? 

• Yes 

• No 

11. [If controller responded ‘Yes’ to Item 10] Did the ATD-1 operations function adequately, 
so that you had a tolerable workload? 

• Yes 

• No 

12. [If controller responded ‘No’ to Item 11] Given that you think the ATD-1 operations were 
not adequate, were the operations controllable? 

• Barely controllable, needs extreme amounts of controller compensation 

• Marginally controllable, needs considerable controller compensation 

• Controllable, needs some controller compensation 

13. [If controller responded ‘Yes’ to Item 11] Are the ATD-1 operations satisfactory without 
improvement? 

• Yes 

• No 

14. [If controller responded ‘No’ to Item 13] How much, if at all, did you have to compensate 
for the tools to make the ATD-1 operations work? (In these situations “compensation” 
means how much did you have to work to counterbalance or offset less desirable actions 
from the tools?) 

• Extensive compensation required to maintain adequate performance 

• Considerable compensation required to maintain adequate performance 

• Moderate compensation required to maintain adequate performance 

15. [If controller responded ‘Yes’ to Item 13] How close to a desired level of performance 
were the ATD-1 operations in this run? 

• Moderate controller compensation needed to reach desired performance 

• Minimal controller compensation required to reach desired performance 

• Realistic ATC emulation performance with no controller correction 
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16. [If controller responded ‘No’ to Item 10] Please describe any events you saw in the last 
run that were unsafe. 


17. Did you have any problems in this run? Please note why and which aircraft were the 
issue. 


18. Did you have to maneuver any non-FIM aircraft to accommodate a FIM-equipped 
aircraft? Please note how many you had to move and describe why. 

• Yes 

• No 

Make a comment on your choice here: 


19. Did you have to change the way you worked to manage the FIM aircraft? 

Please choose all that apply: 

Changed my general scan of the FIM aircraft (compared with non-FIM aircraft) 

Changed the way I monitored separation of the FIM aircraft (compared with non-FIM 
aircraft) 

Changed the way I monitored the FIM aircraft spacing (compared with non-FIM aircraft) 
Issued different types of clearances to the FIM aircraft (compared with non-FIM aircraft) 
Issued more clearances to the FIM aircraft (compared with non-FIM aircraft) 

Considered different types of solutions to FIM aircraft problems (compared with non-FIM 
aircraft) 

No change required 

Other: 
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20. Did you encounter any problems as a result of the target aircraft arriving on a different 
route from its following FIM (self-spacing) aircraft? 

• Yes 

• No 

Make a comment on your choice here: 


21. How manageable was the size of the delay / advance that was required for your non-FIM 
(CMS) aircraft to meet the schedule? 

Very Somewhat Not at all 

manageable manageable manageable 

1 2 3 4 5 6 7 


22. [If controller responded ‘Center North (ZAB 36/43)’ or ‘Center North (ZAB 93/39)’ to 
Item 3] How manageable was the size of the delay / advance that was required for the 
FIM-equipped aircraft to meet the plus or minus 60 sec limit required to issue the FIM 
(self-spacing) clearance? 

Very Somewhat Not at all 

manageable manageable manageable 

1 2 3 4 5 6 7 


23. Which of these available tools did you actively use in the last run? 

Please choose all that apply: 

Meterlist with FIM information 
Delay in the meterlist 
Delay countdown timer 
Runway identifier in data block 
FIM spacing designator 
Route display 
J-rings to 5 NM 
Slot marker 
Speed advisory 
Timeline on your scope 
Spacing cones (bats) 

Other: 
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24. [If controller responded ‘Center North (ZAB 36/43)’ or Center North (ZAB 93/39)’ to 
Item 3] How useful were the CMS tools for managing the FIM aircraft? (e.g., giving you 
information about the FIM aircraft or helping you to space non-FIM aircraft around 
them.) 

Please choose the appropriate response for each item: 



Not useful 
at all 
(ignored) 



Somewhat 



Very useful 




useful 



(essential) 


1 

2 

3 

4 

5 

6 

7 

Delay in the meterlist 

o 

o 

o 

o 

o 

o 

o 

Delay countdown timer 

o 

o 

o 

o 

o 

o 

o 

FIM designators 

o 

o 

o 

o 

o 

o 

o 

Runway identifier 

o 

o 

o 

o 

o 

o 

o 

FIM-specific meterlist information 

o 

o 

o 

o 

o 

o 

o 

All-aircraft meterlist information 

o 

o 

o 

o 

o 

o 

o 


25. [If controller responded ‘Feeder North’ or ‘Final North’ to Item 3] How useful were the 
CMS tools for managing the FIM aircraft (e.g., giving you information about the FIM 
aircraft or helping you to space non-FIM aircraft around them.) 

Please choose the appropriate response for each item: 



Not useful 
at all 
(ignored) 



Somewhat 

useful 



Very useful 
(essential) 


1 

2 

3 

4 

5 

6 

7 

Speed advisories 

o 

o 

o 

o 

o 

o 

o 

Slot markers 

o 

o 

o 

o 

o 

o 

o 

Delay countdown timer 

o 

o 

o 

o 

o 

o 

o 

Timelines 

o 

o 

o 

o 

o 

o 

o 

FIM designators 

o 

o 

o 

o 

o 

o 

o 

Spacing cones (bats) 

o 

o 

o 

o 

o 

o 

o 
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26. Did you issue any of the types of clearances listed below? Please note the aircraft call sign 
(if you can) as well as where and why you took this action in the box on the right. 

Please choose all that apply and provide a comment: 

Suspended FIM spacing 

Resumed FIM spacing 

Decided not to issue a FIM clearance 

Cancelled FIM spacing 

None 

Other: 


27. Did you have to re-issue the IM clearance? If so, please note the ACID(s) and the 
reason(s) why. 

Aircraft 1 (ID and why) 

Aircraft 2 (ID and why) 

Aircraft 3 (ID and why) 

Aircraft 4 (ID and why) 

No, I didn’t have to reissue 


28. How many vectoring maneuvers did you issue to FIM aircraft in this run? (A vector away 
from a route and a second vector back onto the route counts as one maneuver.) 


Before IM spacing 
engaged 

When the FIM-equipped 
aircraft was suspended 
After FIM spacing was 
terminated 


None 

1 or 2 
vector 
maneuvers 

3-5 vector 
maneuvers 

6-10 vector 
maneuvers 

More than 
10 vector 
maneuvers 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

o 
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29. In general, was the FIM pilot’s intent clear? (i.e., Did you have enough information about 
the status of the FIM operations?) 

• Intent / plan was always clear (had enough information) 

• Intent / plan was usually clear 

• Intent / plan was sometimes clear 

• Intent / plan was occasionally clear 

• Intent / plan was not at all / never clear (not enough information) 

Make a comment on your choice here: 


30. Did being responsible for maintaining safe / standard separation for FIM aircraft but not 
managing their spacing affect your monitoring load? 


Monitoring 
load was 
greatly 
reduced 

1 2 


No change 
in 

monitoring 

load 

3 4 5 


Monitoring 
load was 
greatly 
increased 


31. [If controller responded ‘Center North (ZAB 36/43)’ or Center North (ZAB 93/39)’ to 
Item 3] Please rate how accurately the FIM displays (S & ®) reflected the status of the 
FIM aircraft as they entered your sector. 


Very Somewhat 

accurate accurate 

1 2 3 4 5 


Very 

inaccurate 
6 7 


32. [If controller responded ‘Center North (ZAB 36/43)’ or Center North (ZAB 93/39)’ to 
Item 3] Please rate how accurately the FIM displays (S & ®) reflected the status of the 
FIM aircraft as they left your sector. 


Very 

inaccurate 

1 2 


Somewhat 

accurate 

3 4 5 


Very 

accurate 

6 7 N/A 
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33. If applicable, why was the status indicator inaccurate as the FIM aircraft left your sector? 

• I didn’t update the status (please say why) 

• There wasn’t a code for the state of the FIM aircraft 

• The status of the FIM aircraft was changing 

• Other 

Make a comment on your choice here: 


34. Did having FIM aircraft change the amount of coordination you engaged in with the 
controllers upstream and downstream of you? 

No change / 

Much less the same Much more 

coordination level of coordination 

coordination 

1 2 3 4 5 6 7 


35. Was there anything that happened in this run that we forgot to ask about? What would 
you have liked to see happen differently, if anything? 
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Appendix G: Controller Post-Experiment Questionnaire 


1. What is your name? 

**Names will be removed and replaced with coded ID numbers. 

2. Which position did you work in this scenario? 

• Center 

• Feeder/Final 

3. During which year did you retire? 

4. Which aircraft did you pay the most attention to during the experiment? Please note why 
this type of aircraft required more attention from you. 

Please choose all that apply and provide a comment. 

All aircraft equally 

Non-FIM (RNAV) aircraft 

FIM aircraft that were in paired spacing mode 

FIM aircraft that were suspended or terminated 

Other: 


5. Did the speed profile that the FIM aircraft followed fit with your strategy for working 
the non-FIM equipped (RNAV) traffic? Why or why not? 


6. If you had seen a FIM aircraft that was in IM spacing mode and you slowed its lead, what 
would you expect to see the FIM aircraft do? 


7. What was your plan B if the FIM aircraft did not do as you expected in the example 
above? 
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8. To what degree did you accommodate the FIM aircraft? Please note why you selected 
this rating. 

Not at all Sometimes Always 

1 2 3 4 5 6 7 

Make a comment on your choice here: 


9. In what ways would you like to change the FIM operations / aircraft behavior, and why? 


10. Did the winds in this simulation affect the behavior of the FIM aircraft or the FIM tools? 



Always 

Often 

Sometimes 

Occasionally 

Never 

N/A 

FIM aircraft 

o 

o 

o 

o 

o 

o 

FIM spacing 
interval 

o 

o 

o 

o 

o 

o 


11. What impact did the wind variation have on the speed / time advisories and the 
information given by the tools? 


12. Is there a wind condition that could be problematic for the FIM concept? 

• Yes 

• No 

• N/A 

Make a comment on your choice here: 
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13. Please comment on your thoughts of the FIM aircraft sharing routes with the non-FIM 
equipped (RNAV) aircraft. 


14. How did you handle your traffic as it crossed your merge point? 


15. What information would you like to know about the lead aircraft? How would you like 
this information to be displayed? 


16. What procedure would you suggest for coordination when the lead is in a different sector 
from the FIM aircraft? 


17. [If controller responded ‘Center’ to Item 2] This simulation used a procedure where you 
conditioned the FIM traffic to be within a 60 second “window” of their meterfix STA 
before you engaged FIM spacing. Was this possible? If so, how successful was it? 

• Very easy to achieve 60 sec conditioning 

• Easy to achieve 60 sec conditioning 

• 60 sec conditioning was available 

• Hard to achieve 60 sec conditioning 

• Very hard to achieve 60 sec conditioning 

• Impossible to achieve 60 sec conditioning 

Make a comment on your choice here” 
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18. How well did the FIM aircraft conditioning work for you? 


• Conditioning worked very well 

• Conditioning worked well 

• Conditioning sometimes worked 

• Conditioning only worked occasionally 

• Conditioning did not work 

• Impossible to achieve 60 sec conditioning 

Make a comment on your choice here” 


19. How timely / appropriate was the reaction of the system to the speed adjustments of the 
self-spacing (FIM) aircraft? (e.g., When a self-spacing aircraft was flying at a faster speed 
than its target, how quickly did it adjust to a matching speed?) 


Poor 
timing / 
slow 
reaction 

1 2 


Reasonably 

timely 

3 4 5 


Very good 
timing / 
fast 

reaction 
6 7 


20. In addition to standard separation, there is a 0.3 NM buffer. Was the 0.3 NM buffer 
adequate? Please explain. 


• Yes 

• No 

• N/A 


Make a comment on your choice here: 


21. [If controller responded ‘Feeder/Final’ to Item 2] Was the final approach fix the best 
point by which the FIM aircraft should achieve their spacing? 

No, not Yes, 

at all Sometimes always 

1 2 3 4 5 6 7 
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22. [If controller responded ‘Feeder/Final’ to Item 2] Was it clear whether the FIM aircraft 
were going to meet their time at final approach fix targets? 


Never 



Sometimes 



Always 


clear 



clear 



clear 


1 

2 

3 

4 

5 

6 

7 

N/A 


23. [If controller responded ‘Feeder/Final’ to Item 2] Did you modify where you handed off 
your aircraft to the Tower with the final approach fix as the FIM achieve-by point? Was 
it earlier or later than you would have normally made the handoff at PHX? 

• Handoff was much earlier than normal 

• Handoff was earlier than normal 

• Handoff was the same 

• Handoff was later than normal 

• Handoff was much later than normal 


24. Was the transfer of spacing responsibility from you (controller) to the FIM aircraft, and 
back, clear? 

• Yes, always 

• Yes, usually 

• Sometimes 

• Occasionally 

• No 

Make a comment on your choice here: 
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25. When FIM aircraft were in your sector, how acceptable was it for you to be responsible 
for maintaining safe / standard separation between these aircraft (but not managing their 
spacing)? 


Not at all Moderately Very 

acceptable acceptable acceptable 

1 2 3 4 5 


FIM aircraft spacing 


behind FIM aircraft on 
the same route 

o 

o 

o 

o 

o 

FIM aircraft spacing 
behind a FIM aircraft 
on a different route 

o 

o 

o 

o 

o 

FIM aircraft spacing 
behind a CMS aircraft 
on the same route 

o 

o 

o 

o 

o 

FIM aircraft spacing 
behind a CMS aircraft 

o 

o 

o 

o 

o 


on a different route 


26. How would you change flight deck interval management (FIM) to be most helpful to a 
controller? 
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27. Do you think the phraseology you used during the experiment was satisfactory? Please 
refer to the phraseology sheet for exact clearance. 


IM clearance 

Amendments to IM 
clearance 

IM engagement status 
reporting as paired 

Check in with TRACON 

IM operations status 
reporting 

IM clearance suspension 

IM clearance resumption 

Descent clearance 

IM engagement status 
reporting unable 


Not at all 
clear or 
concise 

1 

o 


2 

o 


Moderately 
clear, OK 
length 

3 4 5 


Very 
clear and 
concise 

6 7 N/A 

o o o 


28. Can you suggest changing the phraseology of the FIM clearances in any way to make 
them more clear and / or concise? 
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29. [If controller responded ‘Feeder/Final’ to Item 2] What did you think of the check in by 
the FIM aircraft? How could it be made more concise? Did it give you enough 
information? 

• Always too long 

• Sometimes too long 

• Just the right length 

• Sometimes too short or brief 

• Always too short or brief 

Make a comment on your choice here: 


30. Was the use of the FIM status symbology useful? 

• Yes 

• No 

• N/A 


Make a comment on your choice here: 


31. When FIM operations were involved, were coordination requirements different? If so, 
did you have to coordinate more or less? 


• Yes 

• No 

• N/A 


Make a comment on your choice here: 
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32. Could you have worked the FIM aircraft under current day / baseline conditions, i.e., 
without CMS tools? 


Not at all 

1 2 


Somewhat / 
sometimes 

3 4 5 


Definitely 
6 7 


33. Were there any tips about managing FIM aircraft that you discovered for yourself that 
should be built into training for future studies? 


34. Please rate the usability of the following tools for working the FIM aircraft. “Usability” 
is how logical the functions and button presses associated with the tools are. 



Not at all 



Somewhat 



Very 



usable 



usable 



usable 



1 

2 

3 

4 

5 

6 

7 

N/A 

Slot markers 

o 

o 

o 

o 

o 

o 

o 

o 

Timelines 

o 

o 

o 

o 

o 

o 

o 

o 

Speed advisories 

o 

o 

o 

o 

o 

o 

o 

o 

Early / late indicators 

o 

o 

o 

o 

o 

o 

o 

o 

Spacing designators 

o 

o 

o 

o 

o 

o 

o 

o 

Cones 

o 

o 

o 

o 

o 

o 

o 

o 
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35. Please rate the usability of the following tools for working the FIM aircraft. “Usability” 
is how logical the functions and button presses associated with the tools are. 



Not at all 
usable 



Somewhat 

usable 



Very 

usable 



1 

2 

3 

4 

5 

6 

7 

N/A 

Meterfix STA 

o 

o 

o 

o 

o 

o 

o 

o 

Delay time on aircraft target 

o 

o 

o 

o 

o 

o 

o 

o 

Spacing designators 

o 

o 

o 

o 

o 

o 

o 

o 

Lead aircraft information in 
the meterlist (call sign, 
route) 

o 

o 

o 

o 

o 

o 

o 

o 

FIM delay in the meterlist 

o 

o 

o 

o 

o 

o 

o 

o 

FIM interval to the leader in 
the meterlist 

o 

o 

o 

o 

o 

o 

o 

o 

FIM achieve-by point and 
time in the meterlist 

o 

o 

o 

o 

o 

o 

o 

o 
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36. How confident were you that the controller tools and FIM tools were providing accurate 
information? 



Not at all 
confident 



Moderately 

confident 



Very 

confident 



1 

2 

3 

4 

5 

6 

7 

N/A 

Slot markers 

o 

o 

o 

o 

o 

o 

o 

o 

Timelines 

o 

o 

o 

o 

o 

o 

o 

o 

Speed advisories 

o 

o 

o 

o 

o 

o 

o 

o 

Early / late indicators 

o 

o 

o 

o 

o 

o 

o 

o 

Spacing designators 

o 

o 

o 

o 

o 

o 

o 

o 

Meter STA 

o 

o 

o 

o 

o 

o 

o 

o 

Delay on the aircraft target 

o 

o 

o 

o 

o 

o 

o 

o 

Lead aircraft information in 
meterlist (call sign, route) 

o 

o 

o 

o 

o 

o 

o 

o 

FIM delay in the meterlist 

o 

o 

o 

o 

o 

o 

o 

o 

FIM interval to leader in 
the meterlist 

o 

o 

o 

o 

o 

o 

o 

o 

FIM achieve-by point and 
time in the meterlist 

o 

o 

o 

o 

o 

o 

o 

o 


37. What information should the tools have provided to you that they did not? Please 
consider information presented on your visual display and via voice. 


38. [If controller responded ‘Feeder/Final’ to Item 2] Do you think you could have given 
fewer speed corrections to keep the aircraft in their spacing “slots”? 

• Yes 

• No 
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39. Is there something that we didn’t ask above that you would like to comment on? Please 
note this, or anything you observed in the runs this week that you’d like us to know about. 
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